-- Logs begin at Wed 2022-11-02 13:46:47 UTC, end at Wed 2022-11-02 15:04:36 UTC. -- Nov 02 14:18:19.750844 dv-cinder-862365-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 14:18:19.760190 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 14:18:19 2022] *** Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: nodename: dv-cinder-862365-4-iscsi Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: machine: x86_64 Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: clock source: unix Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: pcre jit disabled Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: detected number of CPU cores: 4 Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: current working directory: / Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: detected binary path: /usr/bin/uwsgi-core Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: your processes number limit is 47680 Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: your memory page size is 4096 bytes Nov 02 14:18:19.762211 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: detected max file descriptor number: 2048 Nov 02 14:18:19.763034 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: lock engine: pthread robust mutexes Nov 02 14:18:19.763034 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: thunder lock: enabled Nov 02 14:18:19.763034 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 14:18:19.763034 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 14:18:19.801641 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: Python main interpreter initialized at 0x56251c7069c0 Nov 02 14:18:19.801641 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: python threads support enabled Nov 02 14:18:19.801641 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: your server socket listen backlog is limited to 100 connections Nov 02 14:18:19.801641 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: your mercy for graceful operations on workers is 90 seconds Nov 02 14:18:19.802053 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 14:18:19.802053 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: *** Operational MODE: preforking *** Nov 02 14:18:19.802347 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: *** uWSGI is running in multiple interpreter mode *** Nov 02 14:18:19.802347 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: spawned uWSGI master process (pid: 81791) Nov 02 14:18:19.802566 dv-cinder-862365-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 14:18:19.803651 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: spawned uWSGI worker 1 (pid: 81802, cores: 1) Nov 02 14:18:19.803651 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: spawned uWSGI worker 2 (pid: 81803, cores: 1) Nov 02 14:18:19.803651 dv-cinder-862365-4-iscsi devstack@keystone.service[81791]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 14:18:24.242430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 14:18:24.242430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: from cryptography.utils import int_from_bytes Nov 02 14:18:24.243523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 14:18:24.243523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: from cryptography.utils import int_from_bytes Nov 02 14:18:24.244003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 14:18:24.244003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: from cryptography.utils import int_from_bytes Nov 02 14:18:24.244154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 14:18:24.244154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: from cryptography.utils import int_from_bytes Nov 02 14:18:24.294489 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 14:18:24.294728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 14:18:24.294849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] command line args: [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 14:18:24.294948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 14:18:24.295092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 14:18:24.295181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 14:18:24.295281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ================================================================================ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 14:18:24.295360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] command line args: [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 14:18:24.295531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] admin_token = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.295616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 14:18:24.295693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ================================================================================ {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 14:18:24.295771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config_dir = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.295943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] admin_token = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] config_source = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] config_dir = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] debug = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] config_source = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.296799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] debug = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297039 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.297928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_config_append = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_config_append = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_dir = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.298952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_dir = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_file = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.299952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.300938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.301852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] publish_errors = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] publish_errors = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.302943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] standard_threads = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.303943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] standard_threads = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_journal = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_json = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.304966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_stderr = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] use_journal = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] use_syslog = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] use_json = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] use_stderr = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] use_syslog = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.305931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 14:18:24.306075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306579 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.306897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.307848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.308956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.309957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.310918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.311940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.312860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.313951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.314904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.315988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316297 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.316952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.317919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318818 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.318964 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.319998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.320110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.320192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.320879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.320971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.321146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.321403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.321544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.321636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.321778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.321900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.322914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.323998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.324995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.325998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.326873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.327908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.328917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.329930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.330980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.331983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.external = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.332913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.external = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333312 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.password = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] auth.token = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.333931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.password = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] auth.token = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.334920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.335921 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.336101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.336693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.336785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.336878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.337190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.337344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.337509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.337650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.337773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.337904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.338914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.339865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.340831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.341864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342104 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.342965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.343225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.343352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.343352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.343587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.343711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.343836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.344910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.345891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.346968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.347944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348579 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.348961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.349884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.350971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.351939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.352750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.353891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.354925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.355909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.356859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.357948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.358872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.359904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.360990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.361925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.362926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.363898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.364784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.365957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.366962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.367080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.367158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.367430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.367604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.367765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.367904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.368969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] role.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] role.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.369994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.370959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.371924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.372906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.373917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.374900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.375904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.376871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.377891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.378998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379421 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.379951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.380900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81802) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.381189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.382130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.382130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.connection = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.382130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.382482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.382482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.382744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.383477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.383623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.383822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.384062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.384303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.384542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.384784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.385035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.385290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.385522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.385766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.385995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.386234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.386466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 14:18:24.386636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81803) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 14:18:24.410526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.410764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.410956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.412434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.412670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.412898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.419576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.421412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.448748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.448976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.450412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.450647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.508349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.508571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.558784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.559028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.559171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.559408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.723418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.723696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.723838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.723977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.724886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.725044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.748812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.749797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.750118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.750254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.750385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.750529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.750660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.750660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.767402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.767631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.780353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.780599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.780766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.780917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81802) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.781847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81803) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 14:18:24.799150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.799371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.799468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 14:18:24.799576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 14:18:24.799685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.799880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 14:18:24.799962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.800109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 14:18:24.800192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.800389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.800490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.800653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.800756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.801043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.801166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.801288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.811368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.811499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.811649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.811856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.811959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.812078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.812208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.812295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.819171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.819279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.823039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.823328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.823472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.823609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.823743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.823904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824356 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.824984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.825194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.825278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.830645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.830769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.830854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.830931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.831006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.831006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.848593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.849191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.850117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.850404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.850684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.850970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.855711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.855964 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.856332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.856421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.856533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.856755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.862948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.863122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.863264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.863349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.870097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.870394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.870493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.870612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.870712 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.870907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.871965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.872232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.872370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.872479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.888927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.889211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.889329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.889437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.893421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.895719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.896873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.897220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.897386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.897681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.901578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.902499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.905053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.905727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.908495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.908806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.909989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.910089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.910454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.910536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.926187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.926503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.926764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.926896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.927027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.927177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.927451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.927713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.934827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.935371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.936396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.936944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.937984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.938262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.938616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.938904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.940893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.941203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.941486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.941617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.941845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.941930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.942130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.942306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.942408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.942812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.954332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.954631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.954886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.959028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.959795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.971542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.972061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.974526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.974981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.978270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.979348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.979787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.980371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.981333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.981615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.981897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.982022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.982182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.982592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.988369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.988979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.990047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.990545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:24.993570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.993977 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.994263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.994478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.994616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.994835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.995136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:24.995415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.002467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:25.002755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.003036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.003309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:25.003605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.003889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.010111 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81802) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:25.010399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.010675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.010948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81803) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 14:18:25.014360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.014986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015112 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81802) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.015677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81803) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 14:18:25.048384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56251c7069c0 pid: 81802 (default app) Nov 02 14:18:25.049303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56251c7069c0 pid: 81803 (default app) Nov 02 14:18:25.051772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446b6b-c29c-45a7-ba0b-2af1dd82db94 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:25.052053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446b6b-c29c-45a7-ba0b-2af1dd82db94 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:25.052329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446b6b-c29c-45a7-ba0b-2af1dd82db94 None None] PATH_INFO: `/v3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:25.057061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Wed Nov 2 14:18:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:33.709360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:33.710201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:33.710201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:33.714861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:18:33.721693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:33.722310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:33.722310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:33.797396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: INFO dbcounter [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] Registered counter for database keystone Nov 02 14:18:33.822310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG oslo_db.sqlalchemy.engines [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81802) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 14:18:33.834018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writer thread running {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 14:18:34.200337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG passlib.handlers.bcrypt [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81802) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 14:18:34.200337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG passlib.handlers.bcrypt [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81802) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 14:18:34.200337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:34.200337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:34.275683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 14:18:34.830837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:34.920129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: INFO dbcounter [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] Registered counter for database keystone Nov 02 14:18:34.941110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG oslo_db.sqlalchemy.engines [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81803) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 14:18:34.948035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writer thread running {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 14:18:35.074083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122f68f-a639-45b0-b876-8666b86dc894 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:35.078598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:35.079235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:35.079515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:35.079785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:35.090091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:35.142852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 14:18:35.143196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720c12f-d15e-4f3c-9359-7e7b6ad6bc58 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:35.154335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Wed Nov 2 14:18:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 14:18:35.160983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:35.180442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137ae1-637b-4e3c-8095-e27f330ed69b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:35.185359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:35.186484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:35.186760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:35.187036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] PATH_INFO: `/v3/projects/e747be529418447d86ba59ccbdb4ada2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:35.188656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] RBAC: Authorizing `identity:get_project(project_id=e747be529418447d86ba59ccbdb4ada2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:35.245109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING keystone.common.rbac_enforcer.enforcer [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 14:18:35.245504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea450a-dc27-4be7-b7d2-3afd419cfa6d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:35.250212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 14:18:35 2022] GET /identity/v3/projects/e747be529418447d86ba59ccbdb4ada2 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:18:37.633312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5f7be-b48f-42a3-862e-fcbedfedddb9 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:37.633913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5f7be-b48f-42a3-862e-fcbedfedddb9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:37.633913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5f7be-b48f-42a3-862e-fcbedfedddb9 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:37.637956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:37.645301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:37.645818 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:37.645903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:37.669994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:37.671536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:37.679898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 14:18:37.968800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5f7be-b48f-42a3-862e-fcbedfedddb9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:37.983100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5f7be-b48f-42a3-862e-fcbedfedddb9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:37.987615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c6f17-7060-48da-99bd-c4ce19bddb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:37.988282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c6f17-7060-48da-99bd-c4ce19bddb64 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:37.988729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c6f17-7060-48da-99bd-c4ce19bddb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:37.989042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c6f17-7060-48da-99bd-c4ce19bddb64 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:37.999209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c6f17-7060-48da-99bd-c4ce19bddb64 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:38.001035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c6f17-7060-48da-99bd-c4ce19bddb64 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:38.011912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Wed Nov 2 14:18:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:18:38.018094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:38.028810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04723ab1-2738-4d27-b01e-1f7a4afccb9d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:38.033812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6db7a-a52a-401a-92a2-0b53e68dbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:38.034552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6db7a-a52a-401a-92a2-0b53e68dbdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:38.034762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6db7a-a52a-401a-92a2-0b53e68dbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:38.035042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6db7a-a52a-401a-92a2-0b53e68dbdd1 admin admin] PATH_INFO: `/v3/users/e928bcf912f045deb4e67843e0dbbcf3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:38.036447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6db7a-a52a-401a-92a2-0b53e68dbdd1 admin admin] RBAC: Authorizing `identity:get_user(user_id=e928bcf912f045deb4e67843e0dbbcf3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:38.038867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6db7a-a52a-401a-92a2-0b53e68dbdd1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:38.041093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Wed Nov 2 14:18:38 2022] GET /identity/v3/users/e928bcf912f045deb4e67843e0dbbcf3 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:40.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a198c78-94c4-41ac-ace7-e6661f6d61b3 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:40.421861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a198c78-94c4-41ac-ace7-e6661f6d61b3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:40.421861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a198c78-94c4-41ac-ace7-e6661f6d61b3 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:40.423163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:40.429407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b03b9-6972-4204-acd6-7b6ec16316c6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:40.447149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b03b9-6972-4204-acd6-7b6ec16316c6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:40.447149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b03b9-6972-4204-acd6-7b6ec16316c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:40.449867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f77b03b9-6972-4204-acd6-7b6ec16316c6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:40.451087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f77b03b9-6972-4204-acd6-7b6ec16316c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:40.541228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 6/11] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:40.548105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca115e-e281-4846-aad8-55e2e4b6d20d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:40.548379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca115e-e281-4846-aad8-55e2e4b6d20d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:40.548649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca115e-e281-4846-aad8-55e2e4b6d20d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:40.550111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 6/12] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:18:40.556014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:40.556298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:40.556581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:40.576760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:40.578014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:40.596679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:40.652915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:40.816682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca115e-e281-4846-aad8-55e2e4b6d20d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:40.931938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.011319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca115e-e281-4846-aad8-55e2e4b6d20d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.060380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba92d4-886e-4cea-92d7-b4c79e0a8b08 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.063795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.064719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.065028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.065390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.081593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:18:41.092651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:18:41.093848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:18:41.094739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:18:41.100143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.109392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.110040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.110367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.110592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.111318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.112785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:41.141975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:41.168645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.sql.core [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 14:18:41.168645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 14:18:41.168645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [parameters: {'id': '26e3a2833d7d488d9287777650d330db', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 14:18:41.168645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81802) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:18:41.195896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result.read() Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 14:18:41.196919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application yield resource Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise self.value Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application session.commit() Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 14:18:41.198033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.session.flush() Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application compat.raise_( Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 14:18:41.199006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 14:18:41.199911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result.read() Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:18:41.200849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [parameters: {'id': '26e3a2833d7d488d9287777650d330db', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 14:18:41.201759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:41.202507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.203348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 8/15] 10.222.0.30 () {60 vars in 1137 bytes} [Wed Nov 2 14:18:40 2022] POST /identity/v3/domains => generated 190 bytes in 266 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:18:41.203994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.228595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:41.236238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb9ce6-7b18-420c-8866-b92f69308b82 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.274027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.274741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.275148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.275400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.276161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:18:41.276463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.277930 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:41.291509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 8/16] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:18:41.297186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.318161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67f2a0-fdd7-41b2-a1e9-031096dc832b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.358509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.359219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.359498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.359803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.371083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.372568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:18:41.390371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:18:41.391476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:18:41.392299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:18:41.392299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:18:41.392299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:41.392299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:18:41.397499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.400219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:41.406528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d8f50-a28c-45ca-8ad3-45d3c896f0cf None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.448698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.449536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.449655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.449894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] PATH_INFO: `/v3/users/e928bcf912f045deb4e67843e0dbbcf3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.451641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] RBAC: Authorizing `identity:get_user(user_id=e928bcf912f045deb4e67843e0dbbcf3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.453881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:41.456214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:18:41 2022] GET /identity/v3/users/e928bcf912f045deb4e67843e0dbbcf3 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:18:41.463330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.486691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:41.525373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca485a-8c77-41a0-aa27-025635d405d8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.562222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.562795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.563087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.563354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.564005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:18:41.564304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.565745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:41.586516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604848fd-6d99-49bb-93c9-051bbeb3f17f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.586714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:18:41.631011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c81d6-4360-45df-9e70-bb8728b3f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.631577 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c81d6-4360-45df-9e70-bb8728b3f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.631859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c81d6-4360-45df-9e70-bb8728b3f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.632256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c81d6-4360-45df-9e70-bb8728b3f9a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.634537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c81d6-4360-45df-9e70-bb8728b3f9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.637401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c81d6-4360-45df-9e70-bb8728b3f9a2 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:41.639594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:18:41.645945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:41.782256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2faf3-51fb-4a7b-af0e-a6f7d510dbca None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:41.814887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:41.825380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8f97-7f54-48e1-bfde-72240f93177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:41.826443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8f97-7f54-48e1-bfde-72240f93177d None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:41.826879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8f97-7f54-48e1-bfde-72240f93177d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:41.827161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8f97-7f54-48e1-bfde-72240f93177d None admin] PATH_INFO: `/v3/domains/default/users/e928bcf912f045deb4e67843e0dbbcf3/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:41.830773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8f97-7f54-48e1-bfde-72240f93177d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e928bcf912f045deb4e67843e0dbbcf3, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:41.830773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8f97-7f54-48e1-bfde-72240f93177d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:42.052950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 11/21] 10.222.0.30 () {58 vars in 1361 bytes} [Wed Nov 2 14:18:41 2022] PUT /identity/v3/domains/default/users/e928bcf912f045deb4e67843e0dbbcf3/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:18:44.964618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07b0fd-f795-4fcd-ae5d-a41ad1fbf4d4 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:44.965449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07b0fd-f795-4fcd-ae5d-a41ad1fbf4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:44.965449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07b0fd-f795-4fcd-ae5d-a41ad1fbf4d4 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:44.966932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:44.973473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dab3ef-f597-4b0c-be29-46b89d436208 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:44.974249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dab3ef-f597-4b0c-be29-46b89d436208 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:44.974249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dab3ef-f597-4b0c-be29-46b89d436208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:44.996418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cfef5-8856-4bee-a56b-e3e0732d2174 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:44.996685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cfef5-8856-4bee-a56b-e3e0732d2174 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:44.997024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cfef5-8856-4bee-a56b-e3e0732d2174 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:44.998968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 12/23] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:45.011260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:45.011887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:45.012355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:45.038125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-03dab3ef-f597-4b0c-be29-46b89d436208 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:45.040147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-03dab3ef-f597-4b0c-be29-46b89d436208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:45.046711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG passlib.handlers.bcrypt [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81803) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 14:18:45.052126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG passlib.handlers.bcrypt [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81803) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 14:18:45.138349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:45.140484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-29969121-fa6e-4f5a-ae2e-a6e5cf265155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:45.195064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:45.220197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 12/24] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:18:45.223040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bff71f-3a77-46cf-ba40-6a0ba2dd6262 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:45.223329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bff71f-3a77-46cf-ba40-6a0ba2dd6262 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:45.223786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bff71f-3a77-46cf-ba40-6a0ba2dd6262 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:45.243184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 13/25] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:45 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:45.255579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa885a-85e0-434e-8739-ba96ad4d14b9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:45.256090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa885a-85e0-434e-8739-ba96ad4d14b9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:45.256605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa885a-85e0-434e-8739-ba96ad4d14b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:45.338632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-62aa885a-85e0-434e-8739-ba96ad4d14b9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:45.345285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-62aa885a-85e0-434e-8739-ba96ad4d14b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:45.403170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 13/26] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:45.524034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 14/27] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:45.772915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10892-ebc7-42a6-b86e-2ef0e00c454a None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:45.773705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10892-ebc7-42a6-b86e-2ef0e00c454a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:45.773859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10892-ebc7-42a6-b86e-2ef0e00c454a None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:45.783744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 14/28] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:45.784229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eabce-7234-48f2-95fc-333173aff251 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:45.784838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eabce-7234-48f2-95fc-333173aff251 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:45.785367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eabce-7234-48f2-95fc-333173aff251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:45.826478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-029eabce-7234-48f2-95fc-333173aff251 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:45.828796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-029eabce-7234-48f2-95fc-333173aff251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:45.917101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10892-ebc7-42a6-b86e-2ef0e00c454a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:45.961498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:46.006992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 15/29] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:18:46.042388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eabce-7234-48f2-95fc-333173aff251 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:46.198848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10892-ebc7-42a6-b86e-2ef0e00c454a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:46.276571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:46.278051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:46.280426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:46.281264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:46.326317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:46.330199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:18:46.367269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:18:46.369035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:18:46.370599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:18:46.370599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:18:46.370599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:18:46.374643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 15/30] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:18:46.376689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:46.399275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eabce-7234-48f2-95fc-333173aff251 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:46.412698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:46.523508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:46.524250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:46.524575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:46.524987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:46.525894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:46.527559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:46.642122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd827-55ed-436c-b134-031eb5370171 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:46.721360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 16/31] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 14:18:46 2022] POST /identity/v3/roles => generated 219 bytes in 680 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:18:46.725165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:46.747450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:46.750364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:46.750928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:46.751417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:46.756782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:46.762966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:46.864383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:47.011131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab00ca-1151-435d-8472-a44a6dea144e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:47.039305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 16/32] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 14:18:46 2022] POST /identity/v3/roles => generated 225 bytes in 664 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:18:47.040611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:47.100069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:47.101603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.101901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.103556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.103556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:18:47.103556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:47.105076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db3f58-8eee-466b-982c-51aca718c8ad None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:47.128103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 17/33] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:47.130917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4137d-183f-4737-8c41-fb59c05ae66f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.132343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4137d-183f-4737-8c41-fb59c05ae66f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.133125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4137d-183f-4737-8c41-fb59c05ae66f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.137548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 18/34] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:47.139289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4137d-183f-4737-8c41-fb59c05ae66f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:47.227319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:47.301042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a71b9-3511-4d82-897d-5894bc24b6f3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:47.373399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:47.380592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4137d-183f-4737-8c41-fb59c05ae66f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:47.382133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5bfa2-9b35-486f-9da9-fb6a8d98648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:47.383584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5bfa2-9b35-486f-9da9-fb6a8d98648a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.384050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5bfa2-9b35-486f-9da9-fb6a8d98648a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.384533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5bfa2-9b35-486f-9da9-fb6a8d98648a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.407212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5bfa2-9b35-486f-9da9-fb6a8d98648a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-bbb5bfa2-9b35-486f-9da9-fb6a8d98648a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:18:47.420792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:18:47.422750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:18:47.424210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 17/35] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:18:47.425469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.425804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.426281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.458107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab329a-e217-4c52-9a38-fb5574c930fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:47.459034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab329a-e217-4c52-9a38-fb5574c930fe None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.459515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab329a-e217-4c52-9a38-fb5574c930fe None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.460350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab329a-e217-4c52-9a38-fb5574c930fe None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.461347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab329a-e217-4c52-9a38-fb5574c930fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:47.463552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab329a-e217-4c52-9a38-fb5574c930fe None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:47.463999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:47.466969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:47.586103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 18/36] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:47.587133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:47.611019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 19/37] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:18:47 2022] POST /identity/v3/projects => generated 301 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:18:47.614072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de601-21ee-4419-b021-0bbaf4106738 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.615643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de601-21ee-4419-b021-0bbaf4106738 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.616480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de601-21ee-4419-b021-0bbaf4106738 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.620101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 20/38] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:47.627727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.628173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.628626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.662083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:47.665113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:47.725028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:47.813300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44438-12e8-4c19-ac93-d7d18779ac8e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:47.814814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 21/39] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:47.877739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:47.879297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:47.880007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:47.880621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:47.882665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:18:47.883588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:47.885962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:47.901137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 19/40] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:18:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:47.908616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:48.037288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:48.073590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:48.108537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480e04b-33c9-4a64-ad0c-f2051b61e345 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:48.174905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:48.174905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:48.174905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:48.174905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] PATH_INFO: `/v3/users/e928bcf912f045deb4e67843e0dbbcf3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:48.175395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] RBAC: Authorizing `identity:get_user(user_id=e928bcf912f045deb4e67843e0dbbcf3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:48.176812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:48.185058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 22/41] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:18:47 2022] GET /identity/v3/users/e928bcf912f045deb4e67843e0dbbcf3 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:48.185058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:48.221718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b66676-0d56-4095-b3f2-cd626c52f7ee None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:48.275101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:48.290768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:48.292633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:48.292633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:48.292938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:48.294493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:48.296531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:48.404364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 20/42] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 14:18:48 2022] POST /identity/v3/roles => generated 223 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:18:48.405443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:48.504314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d3707-5278-4d3c-812c-0bd2c0bb96ed None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:48.565313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:48.566181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:48.566655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:48.567128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:48.568850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:48.571411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:48.573346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 23/43] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:48.582953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:48.618614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:48.630131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e090a09-1908-4745-b39c-b878475e634b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:48.711639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:48.711639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:48.711639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:48.711639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:48.712288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:48.715086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:48.718210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 21/44] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:48.726684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:48.792319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dec621-d001-484d-a5ca-6a8fa369ad59 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:48.823031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:48.845116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2d394-9af5-4170-a8b0-651f17d3e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:48.846087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2d394-9af5-4170-a8b0-651f17d3e6aa None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:48.846574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2d394-9af5-4170-a8b0-651f17d3e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:48.846872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2d394-9af5-4170-a8b0-651f17d3e6aa None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:48.847604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2d394-9af5-4170-a8b0-651f17d3e6aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:48.849280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2d394-9af5-4170-a8b0-651f17d3e6aa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:48.899474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc353a2-1694-4253-b2d0-2c43d18b47ed None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:48.940448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:48.941085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:48.941365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:48.941643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:48.942373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.domain.id=default, user.id=e928bcf912f045deb4e67843e0dbbcf3) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:18:48.942674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:48.944728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310b37-0c48-4089-9702-2c732fafedae None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:48.954664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 22/45] 10.222.0.30 () {56 vars in 1322 bytes} [Wed Nov 2 14:18:48 2022] GET /identity/v3/role_assignments?user.id=e928bcf912f045deb4e67843e0dbbcf3&scope.domain.id=default&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 530 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:18:49.018524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 24/46] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:18:48 2022] POST /identity/v3/projects => generated 312 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:18:51.334621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4ef0a-0d6f-490c-aae5-521afd082379 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:51.335173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4ef0a-0d6f-490c-aae5-521afd082379 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:51.335264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4ef0a-0d6f-490c-aae5-521afd082379 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:51.336564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 23/47] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:51.342311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:51.342908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:51.342908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:51.362870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:51.364085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:51.436785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:51.446434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 25/48] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:51.729909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4ef0a-0d6f-490c-aae5-521afd082379 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:51.854446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4ef0a-0d6f-490c-aae5-521afd082379 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:51.873706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:51.894557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0db99-3ebb-409a-b6bc-0f6b2a82b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:51.895197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0db99-3ebb-409a-b6bc-0f6b2a82b489 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:51.895472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0db99-3ebb-409a-b6bc-0f6b2a82b489 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:51.895743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0db99-3ebb-409a-b6bc-0f6b2a82b489 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:51.896904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0db99-3ebb-409a-b6bc-0f6b2a82b489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:51.898431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0db99-3ebb-409a-b6bc-0f6b2a82b489 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:51.900077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 24/49] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:51.906037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:52.041431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c8e2f-3f83-4ba8-95c6-aa86d568a608 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:52.079985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23a25-68d4-444b-a34e-edc6ad9bd7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:52.080606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23a25-68d4-444b-a34e-edc6ad9bd7e4 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:52.080887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23a25-68d4-444b-a34e-edc6ad9bd7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:52.081195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23a25-68d4-444b-a34e-edc6ad9bd7e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:52.081862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23a25-68d4-444b-a34e-edc6ad9bd7e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:52.083331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23a25-68d4-444b-a34e-edc6ad9bd7e4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:52.206030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 26/50] 10.222.0.30 () {60 vars in 1140 bytes} [Wed Nov 2 14:18:51 2022] POST /identity/v3/projects => generated 298 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:18:54.519371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555676c5-23a1-47cb-ac04-9b31873ce532 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:54.519947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555676c5-23a1-47cb-ac04-9b31873ce532 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:54.519947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555676c5-23a1-47cb-ac04-9b31873ce532 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:54.521289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 25/51] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:18:54.527677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b340d3-dd35-4959-b592-97595c7759df None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:54.528212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b340d3-dd35-4959-b592-97595c7759df None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:54.528212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b340d3-dd35-4959-b592-97595c7759df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:54.548781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-52b340d3-dd35-4959-b592-97595c7759df None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:54.550016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-52b340d3-dd35-4959-b592-97595c7759df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:54.597920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:54.619366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 27/52] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:54.891220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555676c5-23a1-47cb-ac04-9b31873ce532 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:55.000937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555676c5-23a1-47cb-ac04-9b31873ce532 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:55.038138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bdf95-44ba-41d5-b4fd-f960df386838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:55.038770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bdf95-44ba-41d5-b4fd-f960df386838 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:55.039046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bdf95-44ba-41d5-b4fd-f960df386838 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:55.039315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bdf95-44ba-41d5-b4fd-f960df386838 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:55.040574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bdf95-44ba-41d5-b4fd-f960df386838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:55.042068 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bdf95-44ba-41d5-b4fd-f960df386838 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:55.043638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 26/53] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:18:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:18:55.050046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b340d3-dd35-4959-b592-97595c7759df None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:55.158102 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b340d3-dd35-4959-b592-97595c7759df None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:55.193343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82c35-fbb8-405a-8c59-8ac35e039c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:55.194017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82c35-fbb8-405a-8c59-8ac35e039c2d None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:55.194313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82c35-fbb8-405a-8c59-8ac35e039c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:55.194588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82c35-fbb8-405a-8c59-8ac35e039c2d None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:55.195234 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82c35-fbb8-405a-8c59-8ac35e039c2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:55.196680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82c35-fbb8-405a-8c59-8ac35e039c2d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:18:55.304220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 28/54] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:18:55 2022] POST /identity/v3/users => generated 275 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:18:58.859198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221200-2b00-45e1-91df-9e6c972ba472 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:58.859198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221200-2b00-45e1-91df-9e6c972ba472 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:58.859198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221200-2b00-45e1-91df-9e6c972ba472 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:58.860506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 27/55] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:58.866000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d0705-2574-4054-996f-938cef6fbb0e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:58.866495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d0705-2574-4054-996f-938cef6fbb0e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:58.866712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d0705-2574-4054-996f-938cef6fbb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:58.914361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-794d0705-2574-4054-996f-938cef6fbb0e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:58.915599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-794d0705-2574-4054-996f-938cef6fbb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:58.957382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:59.015450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 29/56] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:18:59.067715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9dec9-6200-4d32-a226-c08a836d5afd None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.068348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9dec9-6200-4d32-a226-c08a836d5afd None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.069598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9dec9-6200-4d32-a226-c08a836d5afd None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.071212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 28/57] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:59.077398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11125-502b-4ae6-ba79-1c39384e0afd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.077688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11125-502b-4ae6-ba79-1c39384e0afd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.077965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11125-502b-4ae6-ba79-1c39384e0afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.102252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c4f11125-502b-4ae6-ba79-1c39384e0afd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:59.104124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c4f11125-502b-4ae6-ba79-1c39384e0afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:59.196835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 30/58] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:59.293057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c46af-7343-4909-ae5f-d3be41b42af2 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.293057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c46af-7343-4909-ae5f-d3be41b42af2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.293057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c46af-7343-4909-ae5f-d3be41b42af2 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.294567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 29/59] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:59.300266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.300851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.301365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.320796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:59.322483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:59.418042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c46af-7343-4909-ae5f-d3be41b42af2 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:59.423509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 31/60] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:59.536521 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:59.556191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:59.598335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:59.612917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c46af-7343-4909-ae5f-d3be41b42af2 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:59.650358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f8c5f-6939-424a-b8d1-a53f2f0a09c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:59.650999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f8c5f-6939-424a-b8d1-a53f2f0a09c3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.651282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f8c5f-6939-424a-b8d1-a53f2f0a09c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.651548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f8c5f-6939-424a-b8d1-a53f2f0a09c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.664251 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f8c5f-6939-424a-b8d1-a53f2f0a09c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-aa8f8c5f-6939-424a-b8d1-a53f2f0a09c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:18:59.669240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:18:59.670150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:18:59.670901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:18:59.673291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 30/61] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:18:59.674772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200e80a-9bd3-4ef1-9d4f-372eae8992b0 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.675199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200e80a-9bd3-4ef1-9d4f-372eae8992b0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.675604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200e80a-9bd3-4ef1-9d4f-372eae8992b0 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.677328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:18:59.682706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200e80a-9bd3-4ef1-9d4f-372eae8992b0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:59.683911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77136cc2-2cfe-45a8-bbf4-989835e21161 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:59.730753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba0d0b-db2f-40d7-9144-576677875251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:59.731578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba0d0b-db2f-40d7-9144-576677875251 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.732011 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba0d0b-db2f-40d7-9144-576677875251 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.732405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba0d0b-db2f-40d7-9144-576677875251 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.747250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba0d0b-db2f-40d7-9144-576677875251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9bba0d0b-db2f-40d7-9144-576677875251 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:18:59.756668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:18:59.757771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:18:59.758540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:18:59.759114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 32/63] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:18:59.760391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.760716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.761107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.784198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:18:59.786380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:18:59.893333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 33/64] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:18:59.894745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:18:59.921833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:59.938933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200e80a-9bd3-4ef1-9d4f-372eae8992b0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:18:59.956785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:18:59.984922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:18:59.985644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:18:59.985987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:18:59.986296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:18:59.986980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:18:59.987377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:18:59.989091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.010356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 32/65] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:18:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.010356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.039105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fd3e5-70c0-4d17-9e58-7dedcee874bf None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.108499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.109588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.110089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.110573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.112018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:00.113206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.116683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.133559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 34/66] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.134488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.167361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f72d9e-f251-42a5-bf3a-6caf5a944c5f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.208691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.209862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.210294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.210575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.221926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:00.227322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:00.228795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:00.229660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 33/67] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:00.230448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.235161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:00.259957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ee937-9d44-4e5b-a00d-097ca065ffcd None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.267020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:00.296406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.297183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.297468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.297745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] PATH_INFO: `/v3/users/a7b3a9c09c48450abf3068f29835eb0a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.312462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b3a9c09c48450abf3068f29835eb0a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.314027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.315811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 35/68] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/users/a7b3a9c09c48450abf3068f29835eb0a => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.316628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.339358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce93d9-b0fd-4c54-ae72-f77387d1d18b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.378779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ceb73-a43d-40b9-8ff5-8bdec99443b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.379588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ceb73-a43d-40b9-8ff5-8bdec99443b4 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.380043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ceb73-a43d-40b9-8ff5-8bdec99443b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.380501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ceb73-a43d-40b9-8ff5-8bdec99443b4 None admin] PATH_INFO: `/v3/users/e928bcf912f045deb4e67843e0dbbcf3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.383427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ceb73-a43d-40b9-8ff5-8bdec99443b4 None admin] RBAC: Authorizing `identity:get_user(user_id=e928bcf912f045deb4e67843e0dbbcf3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.385863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ceb73-a43d-40b9-8ff5-8bdec99443b4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.388693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 34/69] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/users/e928bcf912f045deb4e67843e0dbbcf3 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.390077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3b6c8-4c15-4087-b93f-25720d764055 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.390361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3b6c8-4c15-4087-b93f-25720d764055 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.390630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3b6c8-4c15-4087-b93f-25720d764055 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.391978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 35/70] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:00.392701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3b6c8-4c15-4087-b93f-25720d764055 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.439475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da686-0091-4f06-a7d1-0d08e7006e15 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.479186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.479848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.480127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.480398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.483858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:00.493292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:00.498262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:00.499455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:00.500301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 36/71] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:00.501496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.516614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3b6c8-4c15-4087-b93f-25720d764055 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.519645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:00.560030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.560651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.560933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.561226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.561914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:00.562219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.563649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.572931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 36/72] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.573826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.615195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1c791-9021-4fbe-a403-5b798239e3d0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.653242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b9364-22a7-4675-856a-02627eb63559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.654689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9364-22a7-4675-856a-02627eb63559 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.654689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9364-22a7-4675-856a-02627eb63559 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.654689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9364-22a7-4675-856a-02627eb63559 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.655574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b9364-22a7-4675-856a-02627eb63559 None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.657145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b9364-22a7-4675-856a-02627eb63559 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.659069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 37/73] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.659994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.660272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.660541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.680651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:00.681992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:00.688521 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb010-1033-46ea-b873-e4af87a0e1fd None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.723290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.723830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.724128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.724384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.726119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.727950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.729613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 37/74] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.730378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.747764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:00.755684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 38/75] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:00.755684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.773719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:00.874672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd03fe-1841-4ceb-9c8b-ab98afe3262f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.888057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed9ce8-3a37-4a87-84be-51c13af89647 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:00.924026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.925123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.925622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.926105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.926872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:00.927427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.933069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.933233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:00.933870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:00.934055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:00.934304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] PATH_INFO: `/v3/users/a7b3a9c09c48450abf3068f29835eb0a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:00.935683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b3a9c09c48450abf3068f29835eb0a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:00.937755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:00.944347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 39/76] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/users/a7b3a9c09c48450abf3068f29835eb0a => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.944347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:00.955230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 38/77] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:00.955230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.026026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:01.069563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582d51-7743-4aee-808c-b48d75261fa5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.077949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:01.083698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103d9b-faf5-41b6-b786-98bbf81f7e98 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.108895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.109607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.109961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.110177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/a7b3a9c09c48450abf3068f29835eb0a/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.112172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4967f6c093674e39b531bed8fb1ec9b2, user_id=a7b3a9c09c48450abf3068f29835eb0a, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.113682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.122394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.123060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.123343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.123617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/e928bcf912f045deb4e67843e0dbbcf3/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.125612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] RBAC: Authorizing `identity:create_grant(project_id=4967f6c093674e39b531bed8fb1ec9b2, user_id=e928bcf912f045deb4e67843e0dbbcf3, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.127073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.201773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 40/78] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:00 2022] PUT /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/a7b3a9c09c48450abf3068f29835eb0a/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:01.202619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.246243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 39/79] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:00 2022] PUT /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/e928bcf912f045deb4e67843e0dbbcf3/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:01.247774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.409049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf20a-d122-4c24-bc69-806715c3eee1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.423469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234041-f925-4f90-b967-d1f01679d6ba None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.439568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:01.456798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.457854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.458148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.458423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.460668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.462416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.464484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 41/80] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:01 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:01.465446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.474113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.474854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.475140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.475419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] PATH_INFO: `/v3/users/a7b3a9c09c48450abf3068f29835eb0a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.477109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b3a9c09c48450abf3068f29835eb0a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.478982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.481886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 40/81] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:01 2022] GET /identity/v3/users/a7b3a9c09c48450abf3068f29835eb0a => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:01.483786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.522052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:01.625179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930ed6c-73fa-4b7c-9b13-86bc4cf1b873 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.658100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d824fa-5529-4936-8ed3-fd91336dbd31 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.679599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.680203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.680483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.680758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.682521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.684232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.686646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 42/82] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:01.687549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.708878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.710187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.710525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.711153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/a7b3a9c09c48450abf3068f29835eb0a/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.714487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] RBAC: Authorizing `identity:create_grant(project_id=4967f6c093674e39b531bed8fb1ec9b2, user_id=a7b3a9c09c48450abf3068f29835eb0a, role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.716353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.781269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:01.803410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 41/83] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:01 2022] PUT /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/a7b3a9c09c48450abf3068f29835eb0a/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:01.804657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:01.889280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1de16-eb07-44d3-8f5e-516ee76ffea4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:01.979853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:01.982103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:01.985406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:01.986036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] PATH_INFO: `/v3/projects/50c7c199a1f347fda309b358190028c9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:01.987697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] RBAC: Authorizing `identity:get_project(project_id=50c7c199a1f347fda309b358190028c9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:01.992344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:01.996326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 43/84] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:01 2022] GET /identity/v3/projects/50c7c199a1f347fda309b358190028c9 => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:02.005250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:02.029594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:02.069500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a1c82-3d37-4f87-9960-7d28fc584765 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:02.115260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c97073-23af-41af-852b-e4ef2aac0f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:02.116349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97073-23af-41af-852b-e4ef2aac0f1a None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:02.116874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97073-23af-41af-852b-e4ef2aac0f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:02.117372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97073-23af-41af-852b-e4ef2aac0f1a None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:02.118371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c97073-23af-41af-852b-e4ef2aac0f1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:02.122109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c97073-23af-41af-852b-e4ef2aac0f1a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:02.194952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e983f7-aad0-451b-a7c7-7725ab5af155 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:02.230963 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:02.242990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d8286-8ed5-4c42-9a5b-76fcf6cf73fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:02.244159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d8286-8ed5-4c42-9a5b-76fcf6cf73fc None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:02.244659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d8286-8ed5-4c42-9a5b-76fcf6cf73fc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:02.245210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d8286-8ed5-4c42-9a5b-76fcf6cf73fc None admin] PATH_INFO: `/v3/projects/50c7c199a1f347fda309b358190028c9/users/a7b3a9c09c48450abf3068f29835eb0a/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:02.248202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d8286-8ed5-4c42-9a5b-76fcf6cf73fc None admin] RBAC: Authorizing `identity:create_grant(project_id=50c7c199a1f347fda309b358190028c9, user_id=a7b3a9c09c48450abf3068f29835eb0a, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:02.249980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d8286-8ed5-4c42-9a5b-76fcf6cf73fc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:02.279489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 42/85] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:01 2022] POST /identity/v3/users => generated 289 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:02.399023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 44/86] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:02 2022] PUT /identity/v3/projects/50c7c199a1f347fda309b358190028c9/users/a7b3a9c09c48450abf3068f29835eb0a/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:04.725943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3d7f-2a83-452a-9b9c-f7ba89c66df7 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:04.726487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3d7f-2a83-452a-9b9c-f7ba89c66df7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:04.726766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3d7f-2a83-452a-9b9c-f7ba89c66df7 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:04.728533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 43/87] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:04.734683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f14bd-c298-4124-833a-53f70002fa5e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:04.735223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f14bd-c298-4124-833a-53f70002fa5e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:04.735223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f14bd-c298-4124-833a-53f70002fa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:04.766333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5a6f14bd-c298-4124-833a-53f70002fa5e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:04.767620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5a6f14bd-c298-4124-833a-53f70002fa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:04.843107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b74bec-21be-494a-85a9-469fee55b165 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:04.843498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b74bec-21be-494a-85a9-469fee55b165 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:04.843869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b74bec-21be-494a-85a9-469fee55b165 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:04.845468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 44/88] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:19:04.864082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba63d4-872c-4b04-be69-3e122558473e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:04.864587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba63d4-872c-4b04-be69-3e122558473e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:04.865159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba63d4-872c-4b04-be69-3e122558473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:04.928245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-41ba63d4-872c-4b04-be69-3e122558473e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:04.931047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-41ba63d4-872c-4b04-be69-3e122558473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:04.960269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 45/89] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:05.054188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 45/90] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:05.120140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc7e82-48f5-4d6e-8102-e8599bc5854f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:05.124045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc7e82-48f5-4d6e-8102-e8599bc5854f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:05.124045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc7e82-48f5-4d6e-8102-e8599bc5854f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:05.124045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 46/91] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:05.129990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:05.130546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:05.131775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:05.157562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:05.159171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:05.241277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:05.251466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 46/92] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:05.435805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc7e82-48f5-4d6e-8102-e8599bc5854f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:05.659254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc7e82-48f5-4d6e-8102-e8599bc5854f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:05.699762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:05.734394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:05.735459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:05.736127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:05.736331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:05.763676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:05.778796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:05.779819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:05.780637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:05.789074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 47/93] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:05.789161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:05.796645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.093414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7277a2-9d8e-487c-ab6b-a6bee259693f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.105369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51054e5-826c-4197-b1e9-2aa9f95ff763 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.179991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d53b26-25d9-4947-bbe7-78020cb91d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.180568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.181238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.181540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.181837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d53b26-25d9-4947-bbe7-78020cb91d3b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.181944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.182433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d53b26-25d9-4947-bbe7-78020cb91d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.182584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:06.183028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:06.183277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d53b26-25d9-4947-bbe7-78020cb91d3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.184607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:06.196530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d53b26-25d9-4947-bbe7-78020cb91d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:06.197616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 47/94] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:06.198891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-39d53b26-25d9-4947-bbe7-78020cb91d3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:06.203442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:06.204387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:06.205242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:06.207691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 48/95] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:06.209819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8eba2-8f8d-4a3b-9975-f607422f84c8 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.210400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8eba2-8f8d-4a3b-9975-f607422f84c8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.210975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8eba2-8f8d-4a3b-9975-f607422f84c8 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.212776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 49/96] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:06.213955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8eba2-8f8d-4a3b-9975-f607422f84c8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.261233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:06.291198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:06.360429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82021a-6ee4-4174-b869-c41b6d44bff7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.362257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8eba2-8f8d-4a3b-9975-f607422f84c8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.410350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e9031-64bf-4dad-bd35-2bdd8584b271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.411699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e9031-64bf-4dad-bd35-2bdd8584b271 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.412948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e9031-64bf-4dad-bd35-2bdd8584b271 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.413566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e9031-64bf-4dad-bd35-2bdd8584b271 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.421281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.422262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.422780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.423418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] PATH_INFO: `/v3/users/e928bcf912f045deb4e67843e0dbbcf3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.425136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] RBAC: Authorizing `identity:get_user(user_id=e928bcf912f045deb4e67843e0dbbcf3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:06.427005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:06.430889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 50/97] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:06 2022] GET /identity/v3/users/e928bcf912f045deb4e67843e0dbbcf3 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:06.432271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.436447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e9031-64bf-4dad-bd35-2bdd8584b271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-093e9031-64bf-4dad-bd35-2bdd8584b271 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:06.447491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:06.448510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:06.449660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:06.452229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 48/98] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:06.458977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.462441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.463300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.497638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:06.499908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:06.590754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa980-2c47-4e57-85d9-0c7aa9848efa None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.594484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:06.631011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 49/99] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:06.633478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.659930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.660573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.660863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.661174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.661919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:06.664387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:06.665835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:06.691426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 51/100] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:06.692385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.738166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:06.815261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41413da-912f-45b7-8733-51502ae51e51 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.839467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebbe80-0fe6-42cc-a81c-a6402245d9a5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:06.879967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.882773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.883355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.883965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.890525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:06.891242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:06.891710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:06.891995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:06.901029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:06.901029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:06.901029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:06.921565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 52/101] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:06.923375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:06.979605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.017893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:07.019868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:07.091204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3925de-5ebf-4170-ac5b-d9724fd72334 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:07.105057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.146363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e8020-787b-4239-b559-c3f8d8326244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:07.147364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8020-787b-4239-b559-c3f8d8326244 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.147822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8020-787b-4239-b559-c3f8d8326244 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.148287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8020-787b-4239-b559-c3f8d8326244 None admin] PATH_INFO: `/v3/users/a7b3a9c09c48450abf3068f29835eb0a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.150362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e8020-787b-4239-b559-c3f8d8326244 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b3a9c09c48450abf3068f29835eb0a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:07.152843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e8020-787b-4239-b559-c3f8d8326244 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:07.155748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 53/102] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:06 2022] GET /identity/v3/users/a7b3a9c09c48450abf3068f29835eb0a => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:07.158002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e49d9-b0e0-4a06-a029-3dc6d11902ec None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.158409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e49d9-b0e0-4a06-a029-3dc6d11902ec None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.158856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e49d9-b0e0-4a06-a029-3dc6d11902ec None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.161393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 54/103] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:07.164268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e49d9-b0e0-4a06-a029-3dc6d11902ec None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:07.165975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 50/104] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:07.170143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:07.322289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a9be0-0ec6-4ec9-8b8e-a9a823f71af7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:07.324354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.333880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e49d9-b0e0-4a06-a029-3dc6d11902ec None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:07.366309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d3d0-8a3b-48da-b8a0-2355928faefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:07.366309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d3d0-8a3b-48da-b8a0-2355928faefa None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.366869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d3d0-8a3b-48da-b8a0-2355928faefa None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.367021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d3d0-8a3b-48da-b8a0-2355928faefa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.381356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72d3d0-8a3b-48da-b8a0-2355928faefa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:07.381774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:07.382810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.383094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.383386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] PATH_INFO: `/v3/users/a7b3a9c09c48450abf3068f29835eb0a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.384676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b3a9c09c48450abf3068f29835eb0a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:07.386242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-9f72d3d0-8a3b-48da-b8a0-2355928faefa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:07.386624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:07.387707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:07.388798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 51/105] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:07.391168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315f42-992f-4f9a-a27b-8859e44a4188 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.391448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315f42-992f-4f9a-a27b-8859e44a4188 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.391936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315f42-992f-4f9a-a27b-8859e44a4188 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.393120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 55/106] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:07 2022] GET /identity/v3/users/a7b3a9c09c48450abf3068f29835eb0a => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:07.395701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:07.396100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 52/107] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:07.397328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.397739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.398162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.428570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:07.430448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:07.447740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.528613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 53/108] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:07.529546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:07.552139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47d8a6-898a-435d-ad81-54b39149b844 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:07.614196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:07.615695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.616184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.616646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.668565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.684331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78aea7-bbf9-4ec2-8fbf-69d022c866a6 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:07.722827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:07.724469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:07.734622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:07.735347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.735714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.736007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.737376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:07.739197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:07.742841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 54/109] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:07 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:07.748216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:07.766788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.891517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd3669-950d-4b24-a807-ca523e81c9cb None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:07.895926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 56/110] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:07.897602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:07.949120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:07.949890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.950335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.950704 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.951477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:07.952068 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:07.954073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5f11-efd6-4a54-84eb-110488314f6e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:07.969532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 55/111] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:07.971398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:07.971769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:07.972151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:07.983792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:07.999452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:08.000653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:08.029580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0b6d7-124f-4ce3-b1d3-9e75cb1b94b6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.065444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.066050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.066334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.066610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.095147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 56/112] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:08.095147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.095147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:08.153519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.155532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.219370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f48189-9c5d-44a2-ab0a-f6bd66c58a4d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.235186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:08.263481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.264099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.264439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.264719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.265497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=4967f6c093674e39b531bed8fb1ec9b2, user.id=e928bcf912f045deb4e67843e0dbbcf3) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:08.265807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.267314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.276027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 57/113] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/role_assignments?user.id=e928bcf912f045deb4e67843e0dbbcf3&scope.project.id=4967f6c093674e39b531bed8fb1ec9b2&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:08.276870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.298755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 57/114] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:08.299435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.390629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:08.437858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a57818-cfa5-4d97-a640-01921fa9f2ec None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.465577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51d6b7-ff02-4c14-bd55-12ae2ebfc2ab None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.504136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.505487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.506344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.506843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.508916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.512245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.514927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 58/115] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:08.516360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.522531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.523527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.524014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.524294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.525625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.527416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.529094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 58/116] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:08.533028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.575152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:08.655479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bb4cd-1ffc-405f-b588-fb023ce67d92 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.659352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33a36f-d39c-4092-a924-f9b515530fed None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.694349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.695050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.695341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.695615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] PATH_INFO: `/v3/users/a7b3a9c09c48450abf3068f29835eb0a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.696542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.696881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] RBAC: Authorizing `identity:get_user(user_id=a7b3a9c09c48450abf3068f29835eb0a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.697209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.697492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.697769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.698397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.698912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.699999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 59/117] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/users/a7b3a9c09c48450abf3068f29835eb0a => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:08.700387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.700885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.702231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 59/118] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:08.702825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.703821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:08.809610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:08.812682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2825e7-dc3c-4d09-8827-807495f8478d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.822190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24a492-e1c2-4b7e-aba9-6f2c2604e3db None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:08.850437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.851062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.851454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.851729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.852540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:08.854186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:08.862211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:08.862721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:08.863000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:08.863273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:08.877222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:08.882592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:08.888944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 60/119] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:08.889805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.951470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 60/120] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:19:08 2022] POST /identity/v3/projects => generated 302 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:08.952330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:08.965693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.011220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5541e40-f511-47e2-aa62-b48932d168f9 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.064526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.065118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.065408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.065707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.066488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] RBAC: Adding query filter params (role.id=22d32c9fcd36484288e27a5f8630fa18, scope.project.id=4967f6c093674e39b531bed8fb1ec9b2, user.id=a7b3a9c09c48450abf3068f29835eb0a) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:09.066855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.068460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.078126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 61/121] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/role_assignments?user.id=a7b3a9c09c48450abf3068f29835eb0a&scope.project.id=4967f6c093674e39b531bed8fb1ec9b2&role.id=22d32c9fcd36484288e27a5f8630fa18 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:09.085155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:09.108382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0720406-7868-4d3d-9ef3-5c44057c454d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.178387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.179037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.179316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.179799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.223235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.247132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdd1e-027c-4430-afc6-63efc9623ed7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.284080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.291448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.293728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.294015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.294640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.294913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.295225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.295923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] RBAC: Adding query filter params (role.id=95d84bb23f684a9da8785b860247f822, scope.project.id=4967f6c093674e39b531bed8fb1ec9b2, user.id=a7b3a9c09c48450abf3068f29835eb0a) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:09.296447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.298621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.310923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 62/122] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:09 2022] GET /identity/v3/role_assignments?user.id=a7b3a9c09c48450abf3068f29835eb0a&scope.project.id=4967f6c093674e39b531bed8fb1ec9b2&role.id=95d84bb23f684a9da8785b860247f822 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:09.312306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:09.428410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 61/123] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:09.434833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:09.454093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f54a4f-dfc9-4d45-8e84-498230f9a7bd None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.509424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.510053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.510338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.510615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.511305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:09.511612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.513103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.525971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 63/124] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:09.531634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:09.534541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.592494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3838b-5516-4f35-acb5-d4953b8b86ad None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.596148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.641056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.641776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.642057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.642329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] PATH_INFO: `/v3/projects/50c7c199a1f347fda309b358190028c9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.644845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] RBAC: Authorizing `identity:get_project(project_id=50c7c199a1f347fda309b358190028c9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.646759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.649148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 62/125] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:09 2022] GET /identity/v3/projects/50c7c199a1f347fda309b358190028c9 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:09.655116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:09.669159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9fcb-3581-42b2-bcf2-9a740664d617 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.710734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.711697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.712171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.712437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] PATH_INFO: `/v3/users/984acd2a10a1491abe0f9eae3bbfd4f6` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.730467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] RBAC: Authorizing `identity:get_user(user_id=984acd2a10a1491abe0f9eae3bbfd4f6)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.732024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.733897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 64/126] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:09 2022] GET /identity/v3/users/984acd2a10a1491abe0f9eae3bbfd4f6 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:09.739269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:09.776248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a70f-5517-477c-a119-f97e5ca8566b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.801443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.820224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a33862-e026-49b5-b827-de43386e181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.821182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a33862-e026-49b5-b827-de43386e181d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.821678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a33862-e026-49b5-b827-de43386e181d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.821949 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a33862-e026-49b5-b827-de43386e181d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.822753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a33862-e026-49b5-b827-de43386e181d None admin] RBAC: Adding query filter params (role.id=95d84bb23f684a9da8785b860247f822, scope.project.id=50c7c199a1f347fda309b358190028c9, user.id=a7b3a9c09c48450abf3068f29835eb0a) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:09.823002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a33862-e026-49b5-b827-de43386e181d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.824477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a33862-e026-49b5-b827-de43386e181d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.832680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 63/127] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:09 2022] GET /identity/v3/role_assignments?user.id=a7b3a9c09c48450abf3068f29835eb0a&scope.project.id=50c7c199a1f347fda309b358190028c9&role.id=95d84bb23f684a9da8785b860247f822 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:09.857442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59756512-ad1d-4641-bbbe-ca11ce6b52e1 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:09.871964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:09.909900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b56d5-7383-4bd5-9ac6-d87214b9c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:09.910570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b56d5-7383-4bd5-9ac6-d87214b9c5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:09.910851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b56d5-7383-4bd5-9ac6-d87214b9c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:09.911131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b56d5-7383-4bd5-9ac6-d87214b9c5a7 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:09.912478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b56d5-7383-4bd5-9ac6-d87214b9c5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:09.914088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b56d5-7383-4bd5-9ac6-d87214b9c5a7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:09.921265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 65/128] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:09 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:09.926147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a33862-e026-49b5-b827-de43386e181d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:10.067170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a33862-e026-49b5-b827-de43386e181d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:10.109478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6967b8-d9b3-44b9-87a7-b8ddbbedb233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:10.110146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6967b8-d9b3-44b9-87a7-b8ddbbedb233 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:10.110458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6967b8-d9b3-44b9-87a7-b8ddbbedb233 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:10.110731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6967b8-d9b3-44b9-87a7-b8ddbbedb233 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/984acd2a10a1491abe0f9eae3bbfd4f6/roles/d76ead677a6146ed953bcb9d3b38d62f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:10.112802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6967b8-d9b3-44b9-87a7-b8ddbbedb233 None admin] RBAC: Authorizing `identity:create_grant(project_id=4967f6c093674e39b531bed8fb1ec9b2, user_id=984acd2a10a1491abe0f9eae3bbfd4f6, role_id=d76ead677a6146ed953bcb9d3b38d62f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:10.114346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6967b8-d9b3-44b9-87a7-b8ddbbedb233 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:10.189697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 64/129] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:09 2022] PUT /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/users/984acd2a10a1491abe0f9eae3bbfd4f6/roles/d76ead677a6146ed953bcb9d3b38d62f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:11.410917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59c61-272e-4821-bbbc-530f6f11be7a None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:11.411492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59c61-272e-4821-bbbc-530f6f11be7a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:11.411492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59c61-272e-4821-bbbc-530f6f11be7a None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:11.413639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 66/130] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:11.419634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:11.420145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:11.420250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:11.440947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:11.442217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:11.491427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:11.523152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 65/131] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:11.840402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e59c61-272e-4821-bbbc-530f6f11be7a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:11.973493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e59c61-272e-4821-bbbc-530f6f11be7a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:12.018790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629a24-22be-4748-9190-ca8996b0f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:12.019433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629a24-22be-4748-9190-ca8996b0f76e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:12.019711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629a24-22be-4748-9190-ca8996b0f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:12.019979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629a24-22be-4748-9190-ca8996b0f76e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:12.021982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629a24-22be-4748-9190-ca8996b0f76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:12.023487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629a24-22be-4748-9190-ca8996b0f76e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:12.025151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 67/132] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:12.031085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:12.142685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc9173-afbd-465a-adb2-c44a6ea4a902 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:12.183129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67473-07b5-44ee-8dab-eeb9638f8d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:12.183756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67473-07b5-44ee-8dab-eeb9638f8d57 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:12.184014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67473-07b5-44ee-8dab-eeb9638f8d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:12.184283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67473-07b5-44ee-8dab-eeb9638f8d57 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:12.184932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67473-07b5-44ee-8dab-eeb9638f8d57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:12.186501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67473-07b5-44ee-8dab-eeb9638f8d57 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:12.304280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 66/133] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:12 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:12.579648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a014e-328d-4b29-97fb-86c46dfdb9f0 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:12.580161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a014e-328d-4b29-97fb-86c46dfdb9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:12.580502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a014e-328d-4b29-97fb-86c46dfdb9f0 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:12.583157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 68/134] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:12.592995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:12.593877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:12.594065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:12.622755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:12.624642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:12.674849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:12.749483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 67/135] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:13.118169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a014e-328d-4b29-97fb-86c46dfdb9f0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:13.191872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:13.279712 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a014e-328d-4b29-97fb-86c46dfdb9f0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:13.327646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:13.328285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:13.328569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:13.328898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:13.344790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:13.350856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:13.353781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:13.355376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 69/136] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:13.368801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:13.523232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ed821-b813-4bb3-9287-00d8bc74a745 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:13.571933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:13.572574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:13.572836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:13.573146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:13.573837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:13.574144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:13.575551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:13.586814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 68/137] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:13.592598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:13.726090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2906bb-073a-4a4d-9bbc-828430182e45 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:13.736013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:13.772209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:13.773357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:13.773638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:13.773860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] PATH_INFO: `/v3/users/984acd2a10a1491abe0f9eae3bbfd4f6` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:13.775423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] RBAC: Authorizing `identity:get_user(user_id=984acd2a10a1491abe0f9eae3bbfd4f6)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:13.776962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:13.778808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 70/138] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:13 2022] GET /identity/v3/users/984acd2a10a1491abe0f9eae3bbfd4f6 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:13.785075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:13.819958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:13.925479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77b852-d247-4cf3-8214-f12d74e33a89 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:13.971474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:13.972155 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:13.972446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:13.972720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:14.057136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:14.058606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:14.090843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:14.191068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 69/139] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:14.200060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:14.379954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0522-05a4-497d-bc96-f676993257d6 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:14.420307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04c56a-f0a2-4fa7-ad6b-6bef054179a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:14.420942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04c56a-f0a2-4fa7-ad6b-6bef054179a5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:14.421247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04c56a-f0a2-4fa7-ad6b-6bef054179a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:14.421495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04c56a-f0a2-4fa7-ad6b-6bef054179a5 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:14.422676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04c56a-f0a2-4fa7-ad6b-6bef054179a5 None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:14.424130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04c56a-f0a2-4fa7-ad6b-6bef054179a5 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:14.425821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 71/140] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:14 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:14.432575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:14.550382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe89d-a8cf-4cb7-8c5f-27de505b9705 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:14.582584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:14.590416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:14.591023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:14.598792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:14.598792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:14.598792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] RBAC: Adding query filter params (role.id=d76ead677a6146ed953bcb9d3b38d62f, scope.project.id=4967f6c093674e39b531bed8fb1ec9b2, user.id=984acd2a10a1491abe0f9eae3bbfd4f6) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:14.598792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:14.598792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b4887-b26a-44ca-bce2-55d5b829e35c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:14.601424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 70/141] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:14 2022] GET /identity/v3/role_assignments?user.id=984acd2a10a1491abe0f9eae3bbfd4f6&scope.project.id=4967f6c093674e39b531bed8fb1ec9b2&role.id=d76ead677a6146ed953bcb9d3b38d62f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:15.396416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e3c6a-8fbc-4384-b1be-76fcd4560360 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.397250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e3c6a-8fbc-4384-b1be-76fcd4560360 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.397250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e3c6a-8fbc-4384-b1be-76fcd4560360 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.398773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 72/142] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:15.405854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c9d71-08e9-4974-a98b-c3dc15f968f9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.406515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c9d71-08e9-4974-a98b-c3dc15f968f9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.406515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c9d71-08e9-4974-a98b-c3dc15f968f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.429630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-362c9d71-08e9-4974-a98b-c3dc15f968f9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:15.430865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-362c9d71-08e9-4974-a98b-c3dc15f968f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:15.514469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 71/143] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:15.536990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acdc0d-b3b7-4a51-9aaf-01f375ba03bf None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.537310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acdc0d-b3b7-4a51-9aaf-01f375ba03bf None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.537597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acdc0d-b3b7-4a51-9aaf-01f375ba03bf None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.539092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 73/144] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:15.544579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ae9f8-9314-426e-b42a-9d4967a9501d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.544859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ae9f8-9314-426e-b42a-9d4967a9501d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.545176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ae9f8-9314-426e-b42a-9d4967a9501d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.566197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f75ae9f8-9314-426e-b42a-9d4967a9501d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:15.568291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f75ae9f8-9314-426e-b42a-9d4967a9501d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:15.660757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 72/145] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:15.700287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33551-3eeb-4ac1-9b04-4b0252b88abc None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.700582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33551-3eeb-4ac1-9b04-4b0252b88abc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.701070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33551-3eeb-4ac1-9b04-4b0252b88abc None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.702631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 74/146] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:15.709862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.710186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.710463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.731612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:15.733125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:15.784198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:15.793206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33551-3eeb-4ac1-9b04-4b0252b88abc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:15.812590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 73/147] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:15.849775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:15.926667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33551-3eeb-4ac1-9b04-4b0252b88abc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:15.966814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:15.975638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:15.976445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:15.976731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:15.977022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:15.991677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:16.000875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:16.002762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:16.004345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 75/148] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:16.009281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.101271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4458ec-73cf-4951-85bc-e0abdd5436df None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.125629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666948d-63ac-442c-add3-d1a7e110a282 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.127085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:16.141561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.142249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.142557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.142848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.143955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.145454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.147979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 74/149] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:16.148751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.163364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.163970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.164248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.164518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.165295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:16.165701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.167077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.175268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 76/150] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:16.175866 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.200022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:16.261375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea15f-b828-4bc0-b581-857a7f0e71d2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.291653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e2e95-257d-4172-b93f-6d428124a756 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.299444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.300084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.300354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.300626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.311769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:16.316792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:16.318455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 75/151] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:16.319252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.330309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.330941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.331214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.331483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.332130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.334123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.430196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72c80-0477-49a8-9e5b-5afea4a4696c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.432523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:16.457985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 77/152] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:16 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:16.459110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.467650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.468309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.468624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.468870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] PATH_INFO: `/v3/users/fd7a7a088d994e1d953d7b77613a2473` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.485936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] RBAC: Authorizing `identity:get_user(user_id=fd7a7a088d994e1d953d7b77613a2473)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.487012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:16.488903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.492413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 76/153] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/users/fd7a7a088d994e1d953d7b77613a2473 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:16.498594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.593832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1624b-bf5e-446a-80ea-02fb1d6adb9b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.634339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96bb5a-0f9c-48d0-946b-53c46ba6114b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.638560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.639198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.639469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.639743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.640417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:16.640721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.642193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.652162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 78/154] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:16.658515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.690162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.690888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.691171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.691453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.694124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.695656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.697580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 77/155] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:16.703591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.739233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:16.774818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:16.803970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21486e0a-1eb1-45a4-a058-217df2f6477a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.839860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c50954-24ce-4042-8170-66c39e20b31a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:16.865996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.866646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.866983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.867294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] PATH_INFO: `/v3/users/fd7a7a088d994e1d953d7b77613a2473` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.869120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] RBAC: Authorizing `identity:get_user(user_id=fd7a7a088d994e1d953d7b77613a2473)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.870815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.872455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 79/156] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/users/fd7a7a088d994e1d953d7b77613a2473 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:16.878860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:16.888938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:16.889814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:16.890160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:16.890569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/fd7a7a088d994e1d953d7b77613a2473/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:16.892990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] RBAC: Authorizing `identity:create_grant(project_id=20cddc5f50464f6fb97beba61f8a2e3b, user_id=fd7a7a088d994e1d953d7b77613a2473, role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:16.895315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:16.968946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 78/157] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:16 2022] PUT /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/fd7a7a088d994e1d953d7b77613a2473/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:19:17.021636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51157f3a-d8df-46b1-ae42-609b16987187 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:17.067783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee4a4c-bc68-455b-9f3d-80eab42ed400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:17.068919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee4a4c-bc68-455b-9f3d-80eab42ed400 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:17.069447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee4a4c-bc68-455b-9f3d-80eab42ed400 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:17.069995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee4a4c-bc68-455b-9f3d-80eab42ed400 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:17.071703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee4a4c-bc68-455b-9f3d-80eab42ed400 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:17.073681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee4a4c-bc68-455b-9f3d-80eab42ed400 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:17.076116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 80/158] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:16 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:17.086181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:17.208993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:17.223384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7f883-11f3-447d-bd76-f020ac951652 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:17.272065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851043f2-8e54-4d0d-84cc-46ae27c458fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:17.272646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851043f2-8e54-4d0d-84cc-46ae27c458fa None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:17.272918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851043f2-8e54-4d0d-84cc-46ae27c458fa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:17.273220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851043f2-8e54-4d0d-84cc-46ae27c458fa None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/fd7a7a088d994e1d953d7b77613a2473/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:17.275232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851043f2-8e54-4d0d-84cc-46ae27c458fa None admin] RBAC: Authorizing `identity:create_grant(project_id=20cddc5f50464f6fb97beba61f8a2e3b, user_id=fd7a7a088d994e1d953d7b77613a2473, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:17.276690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851043f2-8e54-4d0d-84cc-46ae27c458fa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:17.362285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 79/159] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:17 2022] PUT /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/fd7a7a088d994e1d953d7b77613a2473/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:18.921863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d07cb-3460-4a39-ad77-b2ec184bd485 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:18.922810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d07cb-3460-4a39-ad77-b2ec184bd485 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:18.923287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d07cb-3460-4a39-ad77-b2ec184bd485 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:18.924870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 81/160] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:18.931317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1934e1-5b8e-4af6-9c8c-72a9e2e269b5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:18.932176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1934e1-5b8e-4af6-9c8c-72a9e2e269b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:18.932669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1934e1-5b8e-4af6-9c8c-72a9e2e269b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:18.952345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6a1934e1-5b8e-4af6-9c8c-72a9e2e269b5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:18.953913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6a1934e1-5b8e-4af6-9c8c-72a9e2e269b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:18.985440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982b2ee-c948-43fd-aa48-3fb379e38f93 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:18.986043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982b2ee-c948-43fd-aa48-3fb379e38f93 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:18.986536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982b2ee-c948-43fd-aa48-3fb379e38f93 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:18.988067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 82/161] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:19:18.993460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8300-caca-406b-9d93-f4f89013751b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:18.994080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8300-caca-406b-9d93-f4f89013751b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:18.994547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8300-caca-406b-9d93-f4f89013751b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.030045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-510f8300-caca-406b-9d93-f4f89013751b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:19.031220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-510f8300-caca-406b-9d93-f4f89013751b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:19.047758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 80/162] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:19.048001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:19.107054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 83/163] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:19:19.291768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b0cfe-ace6-4c36-91e0-b38d2a28d4c7 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.291953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b0cfe-ace6-4c36-91e0-b38d2a28d4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.292219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b0cfe-ace6-4c36-91e0-b38d2a28d4c7 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.293623 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 81/164] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:19.301748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.302947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.303438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.322028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:19.323591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:19.328221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b0cfe-ace6-4c36-91e0-b38d2a28d4c7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.405535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 84/165] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:19.406618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.438733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:19.467030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b0cfe-ace6-4c36-91e0-b38d2a28d4c7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:19.507070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:19.513970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:19.514954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.515576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.516008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.531184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:19.537459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:19.538635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:19.539412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:19.540090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a24aa-d0b5-49bd-bb14-863dfe3df799 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:19.542021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 82/166] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:19.548235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.585616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:19.586766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.587152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.587528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.589190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:19.591596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:19.593640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 85/167] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:19.600079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.687265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b619f71-a07d-4585-b02e-0d1fc7da9d1f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:19.725891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:19.726505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.726781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.727051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.727979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:19.728283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:19.729701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:19.734041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51755b2d-4dcb-4d1c-abfc-bdfbb3929119 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:19.738190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 83/168] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:19.738980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.741897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:19.771637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:19.772258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.772534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.772799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.773507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:19.775035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:19.843073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15c0d-5247-47c0-b506-d1dec978fc70 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:19.875904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b33f9-1382-4510-b18c-4284a65ee691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:19.876527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b33f9-1382-4510-b18c-4284a65ee691 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.876803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b33f9-1382-4510-b18c-4284a65ee691 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.877098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b33f9-1382-4510-b18c-4284a65ee691 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.887543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b33f9-1382-4510-b18c-4284a65ee691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d35b33f9-1382-4510-b18c-4284a65ee691 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:19.892787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:19.893868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:19.894708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 84/169] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:19.895365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922145e9-662b-4e69-b9f2-8c34efce07b5 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:19.896418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922145e9-662b-4e69-b9f2-8c34efce07b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:19.896683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922145e9-662b-4e69-b9f2-8c34efce07b5 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:19.898158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 85/170] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:19.898759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922145e9-662b-4e69-b9f2-8c34efce07b5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.933494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 86/171] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:19 2022] POST /identity/v3/users => generated 297 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:19.934561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:19.937179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:19.978082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:20.030108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922145e9-662b-4e69-b9f2-8c34efce07b5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.071229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844288e2-641b-4244-a2c3-92ecbfb8c642 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.071490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bac9f7-0b26-4691-96be-675bc463ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.072350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac9f7-0b26-4691-96be-675bc463ace3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.072779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac9f7-0b26-4691-96be-675bc463ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.073087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac9f7-0b26-4691-96be-675bc463ace3 None admin] PATH_INFO: `/v3/users/6ad22dcbf8134e98ac29ceeee3440a90` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.089334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bac9f7-0b26-4691-96be-675bc463ace3 None admin] RBAC: Authorizing `identity:get_user(user_id=6ad22dcbf8134e98ac29ceeee3440a90)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:20.090986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bac9f7-0b26-4691-96be-675bc463ace3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:20.100261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 86/172] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:19 2022] GET /identity/v3/users/6ad22dcbf8134e98ac29ceeee3440a90 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:20.100261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.100261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.100639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.113324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.114027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.114369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.115728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.116440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:20.116747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:20.118225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:20.122594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:20.124079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:20.128583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 87/173] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:20.129732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.213447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:20.226503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 87/174] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:20.227885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.264809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa200a-7db1-4f83-b9dd-72d402f3b4f6 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.289219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:20.312193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.312807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.313363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.313466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.315670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:20.318397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:20.319812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 88/175] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:20 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:20.325972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.367690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f5aed-ab44-458f-aca7-3f1b08899bcc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.409852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.410082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.410325 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.410602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] PATH_INFO: `/v3/users/fd7a7a088d994e1d953d7b77613a2473` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.413353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] RBAC: Authorizing `identity:get_user(user_id=fd7a7a088d994e1d953d7b77613a2473)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:20.415033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:20.417210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 88/176] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:20 2022] GET /identity/v3/users/fd7a7a088d994e1d953d7b77613a2473 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:20.423803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.450418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5031-b253-4f0b-a176-11094e3d07d6 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.485871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:20.491751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.492594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.493247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.493578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/6ad22dcbf8134e98ac29ceeee3440a90/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.495797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] RBAC: Authorizing `identity:create_grant(project_id=20cddc5f50464f6fb97beba61f8a2e3b, user_id=6ad22dcbf8134e98ac29ceeee3440a90, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:20.497601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:20.544325 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b178-1a44-46f0-a5fb-9849bf44528b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.568238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:20.584816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.585482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.585758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.586027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.586960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 89/177] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:20 2022] PUT /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/6ad22dcbf8134e98ac29ceeee3440a90/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:20.588102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.731037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f33cc-e006-4260-bf37-4592bb20594c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.773064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:20.774191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:20.774569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:20.774996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:20.788786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:20.790048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:20.791547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:20.795102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:20.795987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:20.796795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 90/178] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:19:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:20.806426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.905437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:20.944311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 89/179] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:20.953295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:20.977065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172674-188f-4a4b-9f7c-568893321faf None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:20.991644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:21.074614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:21.075987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:21.076474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:21.076950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:21.077860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:21.078363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:21.080516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:21.096708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 91/180] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:21.103238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:21.163475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832bf9-be86-4be9-b2d4-fe40a12fca57 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:21.259350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:21.259915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:21.260190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:21.260536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:21.261683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:21.263524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:21.269470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 90/181] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:20 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:21.270322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f810eb8-7167-4241-880c-9f3d06e4e8ab None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:21.275426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:21.289038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:21.312971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:21.314087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:21.314591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:21.315062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] PATH_INFO: `/v3/users/fd7a7a088d994e1d953d7b77613a2473` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:21.316417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] RBAC: Authorizing `identity:get_user(user_id=fd7a7a088d994e1d953d7b77613a2473)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:21.318270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:21.325218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 92/182] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:21 2022] GET /identity/v3/users/fd7a7a088d994e1d953d7b77613a2473 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:21.327246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:21.448905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb95f8b-7b62-4e57-9aad-06f42f44133d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:21.476497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:21.501043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f6256-679c-4bff-a8a4-00cfc83a1e01 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:21.521582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:21.522168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:21.522443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:21.522712 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:21.565673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:21.566621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:21.567104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:21.568295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:21.569237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] RBAC: Adding query filter params (role.id=22d32c9fcd36484288e27a5f8630fa18, scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b, user.id=fd7a7a088d994e1d953d7b77613a2473) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:21.570777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:21.572586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:21.587130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 91/183] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:21 2022] GET /identity/v3/role_assignments?user.id=fd7a7a088d994e1d953d7b77613a2473&scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b&role.id=22d32c9fcd36484288e27a5f8630fa18 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:21.626727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:21.653132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:21.654718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:21.787280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 93/184] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 14:19:21.793538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:21.932140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba111c2-a62d-4997-9fa5-33e747832801 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:21.976162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c6cf2-421c-4e6c-a853-3c9afa11947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:21.976687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c6cf2-421c-4e6c-a853-3c9afa11947a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:21.976965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c6cf2-421c-4e6c-a853-3c9afa11947a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:21.977275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c6cf2-421c-4e6c-a853-3c9afa11947a None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:21.978692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c6cf2-421c-4e6c-a853-3c9afa11947a None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:21.980142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c6cf2-421c-4e6c-a853-3c9afa11947a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:21.982121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 92/185] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:21 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:21.987746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:22.096276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:22.118739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0951b-e76f-4309-9a05-486130ae713c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:22.162100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:22.163062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:22.163564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:22.164023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:22.164989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b, user.id=fd7a7a088d994e1d953d7b77613a2473) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:22.165523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:22.167348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ca53a-b8d1-40dc-a1e9-9fdb1b80cac3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:22.175067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 94/186] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:21 2022] GET /identity/v3/role_assignments?user.id=fd7a7a088d994e1d953d7b77613a2473&scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:22.625505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab3304-82ce-4f59-9808-cfab7d9a82b5 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:22.626048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab3304-82ce-4f59-9808-cfab7d9a82b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:22.626048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab3304-82ce-4f59-9808-cfab7d9a82b5 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:22.627575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 93/187] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:22.633603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a5957-0b84-4e95-8358-578f15150a34 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:22.634016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a5957-0b84-4e95-8358-578f15150a34 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:22.634298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a5957-0b84-4e95-8358-578f15150a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:22.655349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-616a5957-0b84-4e95-8358-578f15150a34 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:22.657145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-616a5957-0b84-4e95-8358-578f15150a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:22.733644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 95/188] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:22.771809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6c6f1-7f13-49e3-b747-eb3daca8e126 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:22.772000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6c6f1-7f13-49e3-b747-eb3daca8e126 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:22.772225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6c6f1-7f13-49e3-b747-eb3daca8e126 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:22.773742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 94/189] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:22.779205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:22.779488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:22.779763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:22.798544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:22.799709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:22.872659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 96/190] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:23.002915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6c6f1-7f13-49e3-b747-eb3daca8e126 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.023077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.108523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6c6f1-7f13-49e3-b747-eb3daca8e126 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.148303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.151005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.151695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.151968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.152290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.156647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.165624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:23.172948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:23.173851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:23.174639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 95/191] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:23.181620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.264834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c8c44-c6eb-4037-8da8-193d7ec4cff2 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.276785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.299144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36568d6-11bd-4011-a706-ada31a967ded None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.304906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.305589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.305863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.306127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.307240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:23.308657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:23.310245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 97/192] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:23.315747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.339668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.340539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.340955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.341394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.342057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:23.342354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:23.343761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9470ce-90e6-47c2-b3a1-dce546fa45ab None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:23.353767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 96/193] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:23.355546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37d5a2-a163-4921-9063-47734b33ffd4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.355954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37d5a2-a163-4921-9063-47734b33ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.356330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37d5a2-a163-4921-9063-47734b33ffd4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.357987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 97/194] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:23.359053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37d5a2-a163-4921-9063-47734b33ffd4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.398727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.424473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f09f1-a115-433c-a2ed-63f22cd3c1d3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.460468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.461093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.461371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.461636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.462278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:23.463720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:23.466271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37d5a2-a163-4921-9063-47734b33ffd4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.500160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea5e1-c010-4e39-b1d7-9e7e5ddde0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.500783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea5e1-c010-4e39-b1d7-9e7e5ddde0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.501090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea5e1-c010-4e39-b1d7-9e7e5ddde0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.501375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea5e1-c010-4e39-b1d7-9e7e5ddde0d1 None admin] PATH_INFO: `/v3/users/1daf0102a53a4689b331c8d9c175370c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.512462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.515537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea5e1-c010-4e39-b1d7-9e7e5ddde0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=1daf0102a53a4689b331c8d9c175370c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:23.517022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea5e1-c010-4e39-b1d7-9e7e5ddde0d1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:23.518646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 98/195] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:23 2022] GET /identity/v3/users/1daf0102a53a4689b331c8d9c175370c => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:23.520035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.520035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.521301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.538873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:23.540044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:23.572554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 98/196] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:23 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:19:23.573749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.626505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 99/197] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:23.712754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b32aea-f975-4c59-80fb-10c0ab555749 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.747109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.756416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.757124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.757411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.757694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.758892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:23.760341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:23.762457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 99/198] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:23 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:23.768701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.891967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c58eae-3a8a-44f9-a7d4-cbe457b89e81 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:23.905950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:23.929822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:23.940290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:23.941062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:23.941346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:23.941615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/1daf0102a53a4689b331c8d9c175370c/roles/d76ead677a6146ed953bcb9d3b38d62f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:23.944529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] RBAC: Authorizing `identity:create_grant(project_id=20cddc5f50464f6fb97beba61f8a2e3b, user_id=1daf0102a53a4689b331c8d9c175370c, role_id=d76ead677a6146ed953bcb9d3b38d62f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:23.946246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:24.034054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382a056-8882-421e-a2e4-de43f9872926 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:24.072902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:24.073905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:24.074342 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:24.074793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:24.087471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:24.092836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:24.093983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:24.095019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 100/199] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:24.100827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:24.106272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 100/200] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:23 2022] PUT /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/users/1daf0102a53a4689b331c8d9c175370c/roles/d76ead677a6146ed953bcb9d3b38d62f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:24.187607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:24.245297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132d60e-55c9-4af9-a053-bb3eff918ea5 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:24.293114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:24.294110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:24.294593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:24.295063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:24.296022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:24.296545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:24.298187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:24.309378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 101/201] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:24.315907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:24.439171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6846b71-e10e-4345-a24e-32be4253f43f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:24.481713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:24.482724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:24.483228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:24.483703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] PATH_INFO: `/v3/users/6ad22dcbf8134e98ac29ceeee3440a90` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:24.485172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] RBAC: Authorizing `identity:get_user(user_id=6ad22dcbf8134e98ac29ceeee3440a90)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:24.486955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:24.488891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 101/202] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:24 2022] GET /identity/v3/users/6ad22dcbf8134e98ac29ceeee3440a90 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:24.495479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:24.614879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bd56-3f1a-432d-8367-19f6c86cbcb9 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:24.643409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:24.669036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:24.669931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:24.670295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:24.670767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:24.761185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:24.762832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:24.872928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 102/203] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:24.879093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:24.953052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:25.003071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7f7f8-25eb-454b-ab85-9725a1858864 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:25.041589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f4413-86d2-4aaf-9435-7937ea1e39a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:25.042615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4413-86d2-4aaf-9435-7937ea1e39a9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:25.043084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4413-86d2-4aaf-9435-7937ea1e39a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:25.043569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4413-86d2-4aaf-9435-7937ea1e39a9 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:25.044935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f4413-86d2-4aaf-9435-7937ea1e39a9 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:25.046658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f4413-86d2-4aaf-9435-7937ea1e39a9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:25.048547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 102/204] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:24 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:25.055093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:25.079731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:25.166730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f499d-8b41-4efd-943f-42bbab173954 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:25.208605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:25.210030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:25.210556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:25.211016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:25.211999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] RBAC: Adding query filter params (role.id=95d84bb23f684a9da8785b860247f822, scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b, user.id=6ad22dcbf8134e98ac29ceeee3440a90) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:25.212545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:25.214219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f815ca7-67f9-4f45-a73b-5703ece79b64 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:25.220957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 103/205] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:25 2022] GET /identity/v3/role_assignments?user.id=6ad22dcbf8134e98ac29ceeee3440a90&scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b&role.id=95d84bb23f684a9da8785b860247f822 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:25.984669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679bc8-06e3-4d1c-9b1a-fbca82951682 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:25.985258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679bc8-06e3-4d1c-9b1a-fbca82951682 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:25.985258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679bc8-06e3-4d1c-9b1a-fbca82951682 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:25.986842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 103/206] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:25.993311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e534f0-d355-423d-8658-df242a6fb7a9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:25.993581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e534f0-d355-423d-8658-df242a6fb7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:25.993972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e534f0-d355-423d-8658-df242a6fb7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.013907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c3e534f0-d355-423d-8658-df242a6fb7a9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:26.015127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c3e534f0-d355-423d-8658-df242a6fb7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:26.086920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 104/207] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:26.107514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c234c96-78e9-41bf-86f4-749c933d785a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.107826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c234c96-78e9-41bf-86f4-749c933d785a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.108123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c234c96-78e9-41bf-86f4-749c933d785a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.109607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 104/208] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:26.115400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.115676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.115944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.134557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:26.135725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:26.147092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:26.207381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 105/209] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:26.357331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c234c96-78e9-41bf-86f4-749c933d785a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:26.478399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c234c96-78e9-41bf-86f4-749c933d785a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:26.486252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:26.510985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:26.523811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd9443-b30c-49e7-b834-3018791dc569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:26.524484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd9443-b30c-49e7-b834-3018791dc569 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.524779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd9443-b30c-49e7-b834-3018791dc569 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.525019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd9443-b30c-49e7-b834-3018791dc569 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.538288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd9443-b30c-49e7-b834-3018791dc569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-cbfd9443-b30c-49e7-b834-3018791dc569 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:26.544243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:26.545498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:26.546292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:26.549252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 105/210] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:26.550612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64296f45-c285-41ee-be57-d3bc72d28797 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.550847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64296f45-c285-41ee-be57-d3bc72d28797 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.551121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64296f45-c285-41ee-be57-d3bc72d28797 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.554464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 106/211] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:26.558048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64296f45-c285-41ee-be57-d3bc72d28797 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:26.612734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e93176-1dd8-4c09-a838-9c04be777993 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:26.652662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f96d-aad8-4fe5-9f41-74c545c920f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:26.653317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f96d-aad8-4fe5-9f41-74c545c920f9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.653592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f96d-aad8-4fe5-9f41-74c545c920f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.653859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f96d-aad8-4fe5-9f41-74c545c920f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.654979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1f96d-aad8-4fe5-9f41-74c545c920f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:26.656419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1f96d-aad8-4fe5-9f41-74c545c920f9 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:26.658012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 106/212] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:26.662663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.662663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.662663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.679957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:26.680728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:26.681271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64296f45-c285-41ee-be57-d3bc72d28797 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:26.690296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:26.721757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:26.722369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.722642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.722908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.724096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:26.724421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:26.725911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:26.734802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 107/213] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:26.735612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:26.755879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 107/214] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:26.756774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:26.792891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:26.857470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3029a0-d731-4d7b-9bae-550339ce1ce7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:26.878799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedec86-8869-40cb-8bb9-bdc4a972cdab None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:26.905086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:26.907249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.907524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.907788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.908520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:26.910031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:26.918395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:26.919279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:26.919697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:26.920148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] PATH_INFO: `/v3/users/2f216c24f9da41a3abf0f5218e9d2cb4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:26.933641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:26.936083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] RBAC: Authorizing `identity:get_user(user_id=2f216c24f9da41a3abf0f5218e9d2cb4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:26.937640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:26.939306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 108/215] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:26 2022] GET /identity/v3/users/2f216c24f9da41a3abf0f5218e9d2cb4 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:26.945316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:27.029449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 108/216] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:26 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:27.047800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:27.057827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0c4-a79f-47d9-8ec6-a62e262e028a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:27.104912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:27.105570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:27.105924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:27.106173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:27.107082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] RBAC: Adding query filter params (role.id=95d84bb23f684a9da8785b860247f822, user.id=2f216c24f9da41a3abf0f5218e9d2cb4, scope.system=all) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:27.107275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:27.108731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:27.125045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 109/217] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 14:19:26 2022] GET /identity/v3/role_assignments?user.id=2f216c24f9da41a3abf0f5218e9d2cb4&scope.system=all&role.id=95d84bb23f684a9da8785b860247f822 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:27.188530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c58d4-5f97-454c-9320-6017bd28b111 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:27.190574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:27.236409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:27.237257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:27.237430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:27.237655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:27.253817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:27.259291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:27.260270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:27.261208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 109/218] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:27.267547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:27.345828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:27.393500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e24368-24be-4561-bd28-059378eaccc4 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:27.434886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:27.435506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:27.435784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:27.436054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:27.436729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:27.437125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:27.438436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:27.448795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 110/219] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:27.454936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:27.581078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1212691-2b91-47e4-9714-1e2ed46579ce None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:27.623704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:27.624763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:27.625288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:27.625741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] PATH_INFO: `/v3/users/1daf0102a53a4689b331c8d9c175370c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:27.627154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] RBAC: Authorizing `identity:get_user(user_id=1daf0102a53a4689b331c8d9c175370c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:27.628808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:27.630806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 110/220] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:27 2022] GET /identity/v3/users/1daf0102a53a4689b331c8d9c175370c => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:27.638272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:27.753395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6057-d5d2-499a-9fb7-6c1775713317 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:27.787803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:27.796553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:27.797276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:27.806675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:27.806675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:27.875883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:27.877312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:28.001114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 111/221] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:28.003137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:28.043467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:28.134984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddd6f3-9619-4abc-a4d7-a01b9ddea592 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:28.172224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa108eed-556c-422a-b78f-5169340495d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:28.173510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108eed-556c-422a-b78f-5169340495d8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:28.173792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108eed-556c-422a-b78f-5169340495d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:28.174061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108eed-556c-422a-b78f-5169340495d8 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:28.175222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa108eed-556c-422a-b78f-5169340495d8 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:28.177707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa108eed-556c-422a-b78f-5169340495d8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:28.179752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 111/222] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:28 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:28.188401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:28.221637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:28.301273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25945220-54fb-47c3-b568-f1f17c7d930a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:28.340215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:28.341249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:28.341816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:28.342277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:28.343211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] RBAC: Adding query filter params (role.id=d76ead677a6146ed953bcb9d3b38d62f, scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b, user.id=1daf0102a53a4689b331c8d9c175370c) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:28.343752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:28.345653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b92aa9-aeb7-4099-95b9-f243432797d5 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:28.361158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 112/223] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:19:28 2022] GET /identity/v3/role_assignments?user.id=1daf0102a53a4689b331c8d9c175370c&scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b&role.id=d76ead677a6146ed953bcb9d3b38d62f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:29.336062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca552c56-7aba-49df-b067-dbd58d2dc2c0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.336561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca552c56-7aba-49df-b067-dbd58d2dc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.336561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca552c56-7aba-49df-b067-dbd58d2dc2c0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.337914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 112/224] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:29.344315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df335e-37fc-4189-b8fa-35156530e192 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.344941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df335e-37fc-4189-b8fa-35156530e192 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.344941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df335e-37fc-4189-b8fa-35156530e192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.367743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a1df335e-37fc-4189-b8fa-35156530e192 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:29.368966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a1df335e-37fc-4189-b8fa-35156530e192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:29.456469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 113/225] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:19:29.643984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21b9f2-a992-4f7b-91fc-9f8f31550a68 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.644261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21b9f2-a992-4f7b-91fc-9f8f31550a68 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.644531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21b9f2-a992-4f7b-91fc-9f8f31550a68 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.645998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 113/226] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:29.651612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4491-4bea-4991-8962-901e83d170e0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.651894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4491-4bea-4991-8962-901e83d170e0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.652173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4491-4bea-4991-8962-901e83d170e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.671714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-b93e4491-4bea-4991-8962-901e83d170e0 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:29.672901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b93e4491-4bea-4991-8962-901e83d170e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:29.690523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:29.728747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3bea-523f-4e17-82fb-f02f339492a3 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.729049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3bea-523f-4e17-82fb-f02f339492a3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.729332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3bea-523f-4e17-82fb-f02f339492a3 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.730972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 114/227] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:29.731720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3bea-523f-4e17-82fb-f02f339492a3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:29.753975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 114/228] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:29.758380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.758665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.759075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.779545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:29.781318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:29.845806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:29.851276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3bea-523f-4e17-82fb-f02f339492a3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:29.867159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 115/229] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:29.894987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:29.896248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:29.896593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:29.896764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:29.910445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:29.916161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:29.917666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:29.918469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 115/230] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:29.923621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.026653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.040239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:30.063182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b805c-3f08-4869-a57d-2165c2b85923 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.110636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.111869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.112431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.112863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.114279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:30.114585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:30.116117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:30.133118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 116/231] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:30.138719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.153191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83db5bf-341f-40b4-8f92-c545a14d6bf4 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.189885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.190449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.190667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.190938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.192682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:30.194833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:30.196450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 116/232] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:30.196450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.228756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:30.248652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb281d27-86df-4284-b051-a8a6bd765c60 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.287014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.287655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.287933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.288212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] PATH_INFO: `/v3/users/2f216c24f9da41a3abf0f5218e9d2cb4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.289400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] RBAC: Authorizing `identity:get_user(user_id=2f216c24f9da41a3abf0f5218e9d2cb4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:30.291015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:30.292724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 117/233] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:30 2022] GET /identity/v3/users/2f216c24f9da41a3abf0f5218e9d2cb4 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:30.297829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.307739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:30.313310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b0803-ae4a-42ef-a248-97cd8a2b0983 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.352842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.353637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.353775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.354045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.366150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:30.371009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:30.371935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:30.372754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 117/234] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:30.373566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.405083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0392d7-5c2b-4227-b18e-5f23dd5c1d2d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.443890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.444523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.444906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.445216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] PATH_INFO: `/v3/groups` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.445844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:30.447333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:30.462079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:30.477889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83171a48-419f-41be-9e95-fe0b43a8ec96 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.511480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.512119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.512396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.512667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] PATH_INFO: `/v3/system/users/2f216c24f9da41a3abf0f5218e9d2cb4/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.514103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f216c24f9da41a3abf0f5218e9d2cb4, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:30.515501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:30.561088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 118/235] 10.222.0.30 () {60 vars in 1134 bytes} [Wed Nov 2 14:19:30 2022] POST /identity/v3/groups => generated 229 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:30.561929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.648731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:30.683703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ff44-cce8-415b-9385-63ccc7321a1b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.693787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 118/236] 10.222.0.30 () {58 vars in 1334 bytes} [Wed Nov 2 14:19:30 2022] PUT /identity/v3/system/users/2f216c24f9da41a3abf0f5218e9d2cb4/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:30.728643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.729333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.729726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.729932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:30.730802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:30.731113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:30.732597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:30.749636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 119/237] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:30.756513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:30.959628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169d37a-6686-4a3f-943f-4c394cd43457 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:30.997690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a06c0-7a9f-4fb9-b093-d5240da78758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:30.998332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a06c0-7a9f-4fb9-b093-d5240da78758 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:30.998599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a06c0-7a9f-4fb9-b093-d5240da78758 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:30.998955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a06c0-7a9f-4fb9-b093-d5240da78758 None admin] PATH_INFO: `/v3/users/1f38718dcb7641139ce321389396b2f3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:31.014045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a06c0-7a9f-4fb9-b093-d5240da78758 None admin] RBAC: Authorizing `identity:get_user(user_id=1f38718dcb7641139ce321389396b2f3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:31.015641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a06c0-7a9f-4fb9-b093-d5240da78758 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:31.018721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 119/238] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:30 2022] GET /identity/v3/users/1f38718dcb7641139ce321389396b2f3 => generated 293 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:31.025839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:31.142925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7851d-884f-4ca5-a63d-544b56a066eb None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:31.175051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:31.186972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:31.187994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:31.191133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:31.191133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:31.191133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] RBAC: Adding query filter params (role.id=d76ead677a6146ed953bcb9d3b38d62f, user.id=1f38718dcb7641139ce321389396b2f3, scope.system=all) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:31.191133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:31.191133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ffca8-e5f5-4f07-add4-0c9d66747471 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:31.197367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 120/239] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 14:19:31 2022] GET /identity/v3/role_assignments?user.id=1f38718dcb7641139ce321389396b2f3&scope.system=all&role.id=d76ead677a6146ed953bcb9d3b38d62f => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:32.910104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fbfe3-a5bc-4a16-ad70-ed92a0eb7145 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:32.910764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fbfe3-a5bc-4a16-ad70-ed92a0eb7145 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:32.910764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fbfe3-a5bc-4a16-ad70-ed92a0eb7145 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:32.912451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 120/240] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:32.918742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458710c-8053-4b3b-b980-d90cfb808a35 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:32.919296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458710c-8053-4b3b-b980-d90cfb808a35 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:32.919296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458710c-8053-4b3b-b980-d90cfb808a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:32.939590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2458710c-8053-4b3b-b980-d90cfb808a35 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:32.940786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2458710c-8053-4b3b-b980-d90cfb808a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:33.025602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 121/241] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:33.030556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56084ec-b73b-4890-827e-e54a56deb483 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.030913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56084ec-b73b-4890-827e-e54a56deb483 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.031263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56084ec-b73b-4890-827e-e54a56deb483 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.032715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 121/242] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:33.040162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb9f2-a444-48af-9a31-d82982072e4b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.040640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb9f2-a444-48af-9a31-d82982072e4b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.041181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb9f2-a444-48af-9a31-d82982072e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.072219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a4cbb9f2-a444-48af-9a31-d82982072e4b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:33.074249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a4cbb9f2-a444-48af-9a31-d82982072e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:33.193973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 122/243] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:33.388081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f201b6-e7a0-40f1-ae63-3c98e76268d0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.388477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f201b6-e7a0-40f1-ae63-3c98e76268d0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.388653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f201b6-e7a0-40f1-ae63-3c98e76268d0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.390176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 122/244] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:33.397110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.397397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.397679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.417414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:33.418675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:33.452693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f201b6-e7a0-40f1-ae63-3c98e76268d0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:33.465090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:33.482540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:33.505185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 123/245] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:33.551970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:33.576476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f201b6-e7a0-40f1-ae63-3c98e76268d0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:33.626318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:33.626949 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.627216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.627487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.628722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:33.630626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:33.632245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 123/246] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:33.642442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:33.687685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6772-e5b2-4d50-8ede-6168d968cd9a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:33.728708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:33.729708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.730194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.731145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.732695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:33.745084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:33.750870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:33.751820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:33.752650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 124/247] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:33.759206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:33.760900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfa1d2-3a69-44aa-9b26-1188310a8c06 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:33.805651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:33.806266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.806768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.807046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] PATH_INFO: `/v3/groups` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.807710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:33.809172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:33.810349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:33.866514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1579d-e981-46f6-8710-97f569b22ad0 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:33.909418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 124/248] 10.222.0.30 () {60 vars in 1134 bytes} [Wed Nov 2 14:19:33 2022] POST /identity/v3/groups => generated 226 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:19:33.910255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:33.979959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:33.980934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:33.981547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:33.981826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:33.982515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:33.982861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:33.984342 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:33.997044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 125/249] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:34.003265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:34.064626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165a5b6-e3a1-4636-95a4-55fe7b5e6edc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:34.111813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:34.112767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:34.114231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:34.114231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:34.118837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:34.131879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:34.137708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:34.139423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:34.140230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 125/250] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:34.140882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755401e1-8f82-46d2-a44e-4033b05e7d61 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:34.145179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:34.151399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:34.187037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:34.187924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:34.188207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:34.188480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] PATH_INFO: `/v3/users/2f216c24f9da41a3abf0f5218e9d2cb4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:34.189769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] RBAC: Authorizing `identity:get_user(user_id=2f216c24f9da41a3abf0f5218e9d2cb4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:34.191378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:34.201292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 126/251] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:34 2022] GET /identity/v3/users/2f216c24f9da41a3abf0f5218e9d2cb4 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:34.201292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:34.290846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742fd6f-1b42-4eb6-9ca8-383b7677032b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:34.340781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:34.341429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:34.341781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:34.342046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:34.343243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:34.343546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:34.344958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:34.356152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 126/252] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:34.361968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427e753-5785-443e-9faf-1ff52e58f393 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:34.366588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:34.413497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:34.414112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:34.414387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:34.414711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:34.415447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] RBAC: Adding query filter params (role.id=95d84bb23f684a9da8785b860247f822, user.id=2f216c24f9da41a3abf0f5218e9d2cb4, scope.system=all) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:34.415800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:34.417610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:34.432844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 127/253] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 14:19:34 2022] GET /identity/v3/role_assignments?user.id=2f216c24f9da41a3abf0f5218e9d2cb4&scope.system=all&role.id=95d84bb23f684a9da8785b860247f822 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:34.447285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:34.513554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676a861-f272-4c43-9432-7110b162aa16 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:34.564652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a521d42-74ca-486e-938d-76466e2464d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:34.565606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a521d42-74ca-486e-938d-76466e2464d1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:34.566059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a521d42-74ca-486e-938d-76466e2464d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:34.566479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a521d42-74ca-486e-938d-76466e2464d1 None admin] PATH_INFO: `/v3/users/1f38718dcb7641139ce321389396b2f3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:34.569464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a521d42-74ca-486e-938d-76466e2464d1 None admin] RBAC: Authorizing `identity:get_user(user_id=1f38718dcb7641139ce321389396b2f3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:34.571678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a521d42-74ca-486e-938d-76466e2464d1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:34.573786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 127/254] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:34 2022] GET /identity/v3/users/1f38718dcb7641139ce321389396b2f3 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:34.580772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:34.634743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:34.728727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ff082-d45f-4a7b-8194-2285287d38ee None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:34.773318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ad96-561f-425d-a95b-be11c1f41000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:34.773988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ad96-561f-425d-a95b-be11c1f41000 None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:34.774270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ad96-561f-425d-a95b-be11c1f41000 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:34.774567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ad96-561f-425d-a95b-be11c1f41000 None admin] PATH_INFO: `/v3/system/users/1f38718dcb7641139ce321389396b2f3/roles/d76ead677a6146ed953bcb9d3b38d62f` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:34.776284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8ad96-561f-425d-a95b-be11c1f41000 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f38718dcb7641139ce321389396b2f3, role_id=d76ead677a6146ed953bcb9d3b38d62f)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:34.777738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8ad96-561f-425d-a95b-be11c1f41000 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:34.860093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 128/255] 10.222.0.30 () {58 vars in 1334 bytes} [Wed Nov 2 14:19:34 2022] PUT /identity/v3/system/users/1f38718dcb7641139ce321389396b2f3/roles/d76ead677a6146ed953bcb9d3b38d62f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:37.264471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14c4dd-e8e9-40e6-b122-0ec0b51241a2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.265491 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14c4dd-e8e9-40e6-b122-0ec0b51241a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.266020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14c4dd-e8e9-40e6-b122-0ec0b51241a2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.267954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 128/256] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:37.274880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5eae5-87c7-4f71-85c0-19165c7325df None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.275379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5eae5-87c7-4f71-85c0-19165c7325df None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.275461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5eae5-87c7-4f71-85c0-19165c7325df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.309956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d0b5eae5-87c7-4f71-85c0-19165c7325df None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:37.311212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d0b5eae5-87c7-4f71-85c0-19165c7325df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:37.403567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 129/257] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 14:19:37.464101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ddae0-353a-4ba5-9de3-4a23d9cf5cca None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.464495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ddae0-353a-4ba5-9de3-4a23d9cf5cca None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.464780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ddae0-353a-4ba5-9de3-4a23d9cf5cca None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.466160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 129/258] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:37.477711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6d0a-a89f-4468-80f6-4d6a3e5449fb None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.478383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6d0a-a89f-4468-80f6-4d6a3e5449fb None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.478854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6d0a-a89f-4468-80f6-4d6a3e5449fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.509593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a37c6d0a-a89f-4468-80f6-4d6a3e5449fb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:37.511158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a37c6d0a-a89f-4468-80f6-4d6a3e5449fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:37.524035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 130/259] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 14:19:37.568587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327e1dd-442e-4afc-b9d8-f1b3894438b4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.568954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327e1dd-442e-4afc-b9d8-f1b3894438b4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.569294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327e1dd-442e-4afc-b9d8-f1b3894438b4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.570953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 130/260] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:37.577105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.577948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.578672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.616131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:37.617968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:37.633321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 131/261] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 14:19:37.791785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327e1dd-442e-4afc-b9d8-f1b3894438b4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:37.808663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327e1dd-442e-4afc-b9d8-f1b3894438b4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:37.813097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:37.813735 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.814013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.814277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.833402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:37.838569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:37.839518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:37.840332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 131/262] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 14:19:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:37.846143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:37.864329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a697c-28f5-4ba0-a54b-91d60c37993a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:37.870112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:37.871039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.871548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.872017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.872878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:37.873458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:37.875656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:19:37.875656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:19:37.875656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:19:37.877048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:37.893096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 132/263] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 14:19:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:37.905543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:37.919762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151455cd-657e-4c2c-a5d7-6fe5a984f448 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:37.928499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:37.929183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.929460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.929732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.942285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:37.944656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:19:37.944656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:19:37.944656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:19:37.946037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:37.948353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 132/264] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 14:19:37 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:37.954053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:37.969174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba4d54-451d-4364-9706-7921b3bb2704 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:37.975100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:37.975756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:37.976143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:37.976619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:37.978539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:37.980826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:37.982926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 133/265] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 14:19:37 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:37.989817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.002493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd80e-0126-4c1f-b4cd-f6e2b0abd671 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.008516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.009299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.009613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.010061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/groups/a1569170bb5f4439af64129064c70364/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.012422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4967f6c093674e39b531bed8fb1ec9b2, group_id=a1569170bb5f4439af64129064c70364, role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.013497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:19:38.013497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:19:38.013497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:19:38.015468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.099623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.112825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d2a7-ce31-4e2a-a9a3-3c7da0e17586 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.120334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.121320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.121581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.121865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.146558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:38.154215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:38.156565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:38.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 134/266] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:38.160470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 133/267] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 14:19:37 2022] PUT /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/groups/a1569170bb5f4439af64129064c70364/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:38.160722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.166106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.175389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5f41-8054-46da-ad33-2d9dfaa29663 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.193434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:38.258028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255af64f-f5f3-48de-9fc9-7a23392f3435 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.263710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.264390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.264674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.265254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.266242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:38.266560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.267607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:19:38.267607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:19:38.267607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:19:38.269020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.281093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 134/268] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:38.292494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.304058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.305130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40444d-cd70-42d1-b2cd-1ca4bdaff767 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.305539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.306075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.306487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.310646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.311569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.311976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.312361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.314309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.317113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.319700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 135/269] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:38.326535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.333073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:38.339117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:38.340088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:38.340936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:38.341597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3a10-57d4-435b-9f6c-be4b1dae9e8d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.345414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 135/270] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:38.348364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.353317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.353317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.353317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.354137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.356473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.358680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 136/271] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:38.360281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.364815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.376420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7048a-820f-42fb-9e79-06882edbd59a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.377717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28c9e-35e3-4d4b-b5b1-11b664dc1024 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.382895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.383592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.383847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.384100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/groups/a1569170bb5f4439af64129064c70364/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.386050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.386195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4967f6c093674e39b531bed8fb1ec9b2, group_id=a1569170bb5f4439af64129064c70364, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.387105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.387694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.388174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.389069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.389454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:38.389981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.392189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.409978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 136/272] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:38.416600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.431246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7118762-1a6a-42ac-b3e8-015d1c13da18 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.440309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.441469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.442015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.442506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.449973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.465744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:19:38.465744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:19:38.465744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:19:38.465744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.465744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 137/273] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:38.465744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.479698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90c8af-de12-4f81-95ae-36767bf5e9a1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.490129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813f82b-d96d-4c9f-a129-d07586e047b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.491335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f82b-d96d-4c9f-a129-d07586e047b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.491856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f82b-d96d-4c9f-a129-d07586e047b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.492348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f82b-d96d-4c9f-a129-d07586e047b8 admin admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.497211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813f82b-d96d-4c9f-a129-d07586e047b8 admin admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.499721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813f82b-d96d-4c9f-a129-d07586e047b8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.501052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 137/274] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 14:19:38 2022] PUT /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2/groups/a1569170bb5f4439af64129064c70364/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:19:38.505398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 138/275] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 14:19:38 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:38.510648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:38.545805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:38.613160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72162a99-a474-43f2-ac35-1e3f5ff267d6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:38.620230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380532b-7648-4598-8e36-a8e64777a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:38.621024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380532b-7648-4598-8e36-a8e64777a18f admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.621317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380532b-7648-4598-8e36-a8e64777a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.621583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380532b-7648-4598-8e36-a8e64777a18f admin admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/groups/a1569170bb5f4439af64129064c70364/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.623858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380532b-7648-4598-8e36-a8e64777a18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cddc5f50464f6fb97beba61f8a2e3b, group_id=a1569170bb5f4439af64129064c70364, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:38.626763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380532b-7648-4598-8e36-a8e64777a18f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:38.715969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 138/276] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 14:19:38 2022] PUT /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/groups/a1569170bb5f4439af64129064c70364/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:38.920346 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9025f-c9c6-42a9-b172-e00ef13cb30e None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.920694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9025f-c9c6-42a9-b172-e00ef13cb30e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.920873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9025f-c9c6-42a9-b172-e00ef13cb30e None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.922692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 139/277] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:38.934828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e7ad-8439-4bff-b64c-0c49e3ea329d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.935177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e7ad-8439-4bff-b64c-0c49e3ea329d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.935379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e7ad-8439-4bff-b64c-0c49e3ea329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.969038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5d57e7ad-8439-4bff-b64c-0c49e3ea329d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:38.970368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5d57e7ad-8439-4bff-b64c-0c49e3ea329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:38.972332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f72070-6122-49cd-9ea9-afa1bdaec335 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.972598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f72070-6122-49cd-9ea9-afa1bdaec335 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.972964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f72070-6122-49cd-9ea9-afa1bdaec335 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:38.974646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 140/278] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:19:38.985728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:38.986058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:38.986285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.030139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:39.031390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:39.077702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 139/279] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 14:19:39.137809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980d46c-4dc8-4824-b5ec-2ee5c26f183d None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.138149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980d46c-4dc8-4824-b5ec-2ee5c26f183d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.138340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980d46c-4dc8-4824-b5ec-2ee5c26f183d None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.140070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 140/280] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:39.147330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.147696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.147866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.177893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:39.179002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:39.194664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 141/281] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 14:19:39.260568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 141/282] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:19:39.464010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.474933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4686f-73dd-4a66-aa12-3c646040c41d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:39.482240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:39.482891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.483172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.483446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.497098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:39.502451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:39.503640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:39.505429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:39.507076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 142/283] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:39.512743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.525829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa86535-c9b0-4d9f-b8a7-aafc014c8718 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:39.533348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:39.533980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.534265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.534585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.535309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:39.535745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:39.538221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:39.555219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 142/284] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:39.560300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.572159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.579749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d278cc7-2b04-4a85-9f7a-8bfb12715e3d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:39.586244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:39.587001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.587388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.587838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.592337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:39.595103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:39.599823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 143/285] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:39.602201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:39.605068 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.621064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7af50-58d7-4e08-9df2-fe49d6646ffc admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:39.630685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:39.631487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.631791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.632070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.633570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:39.635982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:39.638658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 144/286] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:39.647727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.661815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a38cb-11cf-47f3-9c8c-39b92dba687c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:39.667421 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:39.677345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.677345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.677345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/groups/a1569170bb5f4439af64129064c70364/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.677345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cddc5f50464f6fb97beba61f8a2e3b, group_id=a1569170bb5f4439af64129064c70364, role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:39.678004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:19:39.678004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:19:39.678004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:19:39.678004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:39.703055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baf946-3130-479f-b119-0d4a6c75c50c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:39.747274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:39.747835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:39.748166 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:39.748442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:39.763363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:39.768619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:39.771812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:39.773857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 143/287] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:39.779352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.839019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 145/288] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 14:19:39 2022] PUT /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b/groups/a1569170bb5f4439af64129064c70364/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:19:39.839872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:39.951176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:39.972538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:39.997629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b9f0-c8f9-4b64-8eee-1aadd0938128 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.008233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf0d2-2dff-42b7-9397-34ff76f50ffd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.018175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.018940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.019216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.023455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.040522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:40.051181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:40.053581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:40.055151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 146/289] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:40.061505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:40.101711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.101711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.101711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.101711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.102054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:40.102054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.102054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.102238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ead6ec-aa5c-4a34-8c57-45561e612e64 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.102307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 144/290] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:40.102373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.102714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:40.102796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.102796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.102796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.102796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:40.102796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.102796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.118186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 147/291] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:40.128143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:40.147999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7cb1c-2c06-45ba-b0eb-01f5074e46c4 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.162029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.163052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.163529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.164014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] PATH_INFO: `/v3/groups/345178b7e9eb48f3b98eb7bd8a274719` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.178806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] RBAC: Authorizing `identity:get_group(group_id=345178b7e9eb48f3b98eb7bd8a274719)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.181262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.183820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 148/292] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 14:19:40 2022] GET /identity/v3/groups/345178b7e9eb48f3b98eb7bd8a274719 => generated 229 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:40.189417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:40.207959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e5b4f-5646-437f-a2d8-fbb5baf7b864 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.219817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.220480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.221965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.222251 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] PATH_INFO: `/v3/projects/e747be529418447d86ba59ccbdb4ada2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.228653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] RBAC: Authorizing `identity:get_project(project_id=e747be529418447d86ba59ccbdb4ada2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.230645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.235722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 149/293] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 14:19:40 2022] GET /identity/v3/projects/e747be529418447d86ba59ccbdb4ada2 => generated 344 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:40.242548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:40.265201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0df04-f96f-4ea1-b40c-6a70608c01bb admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.274883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.275762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.276041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.276314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] PATH_INFO: `/v3/projects/e747be529418447d86ba59ccbdb4ada2/groups/345178b7e9eb48f3b98eb7bd8a274719/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.283685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e747be529418447d86ba59ccbdb4ada2, group_id=345178b7e9eb48f3b98eb7bd8a274719, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.289558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.313442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5196be-491b-4257-9226-915f640dc1e3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.359986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1cafd-f7e6-4602-840e-7dabcf7ae449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.360763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cafd-f7e6-4602-840e-7dabcf7ae449 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.361143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cafd-f7e6-4602-840e-7dabcf7ae449 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.361441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cafd-f7e6-4602-840e-7dabcf7ae449 None admin] PATH_INFO: `/v3/users/1f38718dcb7641139ce321389396b2f3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.362617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1cafd-f7e6-4602-840e-7dabcf7ae449 None admin] RBAC: Authorizing `identity:get_user(user_id=1f38718dcb7641139ce321389396b2f3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.371966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 150/294] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 14:19:40 2022] PUT /identity/v3/projects/e747be529418447d86ba59ccbdb4ada2/groups/345178b7e9eb48f3b98eb7bd8a274719/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:19:40.372165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1cafd-f7e6-4602-840e-7dabcf7ae449 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.374022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 145/295] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:19:40 2022] GET /identity/v3/users/1f38718dcb7641139ce321389396b2f3 => generated 293 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:40.380714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:40.513296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8dd14-0a02-4833-80bc-379c6d53cc5d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:40.555449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:40.556035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:40.556372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:40.556702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:40.557472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] RBAC: Adding query filter params (role.id=d76ead677a6146ed953bcb9d3b38d62f, user.id=1f38718dcb7641139ce321389396b2f3, scope.system=all) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:40.557779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:40.559307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b00532-203b-49c4-b062-34ab5ba6518e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:40.573415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 151/296] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 14:19:40 2022] GET /identity/v3/role_assignments?user.id=1f38718dcb7641139ce321389396b2f3&scope.system=all&role.id=d76ead677a6146ed953bcb9d3b38d62f => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:41.892839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd784e4-66ed-4be7-b38c-41117d99b2fd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:41.893423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd784e4-66ed-4be7-b38c-41117d99b2fd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:41.893423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd784e4-66ed-4be7-b38c-41117d99b2fd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:41.894927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 146/297] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:41.900820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ffff2-793f-4a47-9532-8bee68d72166 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:41.901810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ffff2-793f-4a47-9532-8bee68d72166 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:41.902281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ffff2-793f-4a47-9532-8bee68d72166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:41.918240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:41.924017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-939ffff2-793f-4a47-9532-8bee68d72166 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:41.926493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-939ffff2-793f-4a47-9532-8bee68d72166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:42.005027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 152/298] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:42.295480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd784e4-66ed-4be7-b38c-41117d99b2fd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:42.314523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:42.430725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccbd19-25b2-43e0-a2c6-d456342ded9b None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:42.431372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccbd19-25b2-43e0-a2c6-d456342ded9b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:42.431881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccbd19-25b2-43e0-a2c6-d456342ded9b None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:42.434281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 153/299] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:42.450226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd784e4-66ed-4be7-b38c-41117d99b2fd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:42.451129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae86f-9497-48be-9f5a-f3c51425c562 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:42.451612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae86f-9497-48be-9f5a-f3c51425c562 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:42.452102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae86f-9497-48be-9f5a-f3c51425c562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:42.480179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-049ae86f-9497-48be-9f5a-f3c51425c562 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:42.482076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-049ae86f-9497-48be-9f5a-f3c51425c562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:42.510561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a96426-9ae0-48ed-b62b-a92282b4ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:42.511629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a96426-9ae0-48ed-b62b-a92282b4ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:42.512125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a96426-9ae0-48ed-b62b-a92282b4ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:42.512596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a96426-9ae0-48ed-b62b-a92282b4ac4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:42.525633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a96426-9ae0-48ed-b62b-a92282b4ac4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-42a96426-9ae0-48ed-b62b-a92282b4ac4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:42.531461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:42.532594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:42.533640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:42.536318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 147/300] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:42.537879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a6571-2cb8-425b-8ebb-97bb1069a502 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:42.543045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a6571-2cb8-425b-8ebb-97bb1069a502 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:42.543045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a6571-2cb8-425b-8ebb-97bb1069a502 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:42.543045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 148/301] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:42.548219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a6571-2cb8-425b-8ebb-97bb1069a502 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:42.587734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 154/302] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:19:42.590198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:42.590756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:42.591250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:42.634072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:42.635757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:42.702196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:42.762324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:42.770430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a6571-2cb8-425b-8ebb-97bb1069a502 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:42.777763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 155/303] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:42.820768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:42.822194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:42.822476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:42.822745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:42.823428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:42.823733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:42.825157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba8d6-0c19-4999-a8ca-d2435b6e9d35 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:42.836916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 149/304] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:42.842846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:42.979273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb533e-e559-441c-8de8-fac0b3d0a00a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.018921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.020240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.021228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.021228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.032967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832ddac-24a9-4860-8bc8-49841d657884 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.033837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832ddac-24a9-4860-8bc8-49841d657884 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.033837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832ddac-24a9-4860-8bc8-49841d657884 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.034965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 150/305] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:43.041259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.041620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.041889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.062385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:43.063690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:43.099778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:43.107702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:43.109241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:43.173036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 151/306] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:19:43.173036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.274750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 156/307] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:43.303934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.332533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:43.403299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c413-b48b-4ef2-9712-2df08b51f113 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.452416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:43.456435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.458166 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.458643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.459260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.464142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231530-1b15-44d9-80f7-4474b152d56e None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.476034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:43.483256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:43.484470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:43.485552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:43.489808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 152/308] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:43.491507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.514683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.515398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.515765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.516140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.517433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:43.519129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:43.522289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 157/309] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:43.523536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.642326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7bfe-22b8-4c8e-b6a8-765dbe8884b3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.673911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e461cc-d2be-45ba-abca-210961dd8cb0 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.674521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:43.707378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.707378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.707378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.707378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.732303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:43.732507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.732965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.733281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.733555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.734234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:43.734535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:43.736059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:43.737925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:43.739524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:43.740318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 153/310] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:43.744373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.761201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 158/311] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:43.763358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.809966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:43.882736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717010-7d14-4dad-b7eb-6c0b515ccdf5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.915360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806407f7-29da-4616-a5e1-a71446be3f01 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:43.935060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.935978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.935978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.936257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.937481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:43.938941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:43.940725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 154/312] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:43.946963 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:43.963305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:43.963966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:43.964413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:43.964687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:43.981997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:43.987187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:43.988727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:43.989630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 159/313] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:43.993734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.022756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.085638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce2b72-5648-4c3a-88df-9dccf9872288 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.130860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.137547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:44.137788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.138091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.138364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.139044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:44.139340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:44.140767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd718f-9905-4201-96f5-66fee30ab4b8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:44.148390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9647-4f65-4ff4-99ee-aba3fddb25f5 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.151939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 155/314] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:44.153537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c225-c06e-4414-8610-39cc042e0d1a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.154001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c225-c06e-4414-8610-39cc042e0d1a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.154430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c225-c06e-4414-8610-39cc042e0d1a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.155892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 156/315] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:44.156821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80c225-c06e-4414-8610-39cc042e0d1a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.208504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:44.209687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.209992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.210243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.300897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80c225-c06e-4414-8610-39cc042e0d1a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.307771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:44.309648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:44.317058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.350609 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:44.351227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.351501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.351870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.352745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] RBAC: Adding query filter params (group.id=a1569170bb5f4439af64129064c70364, role.id=22d32c9fcd36484288e27a5f8630fa18, scope.project.id=4967f6c093674e39b531bed8fb1ec9b2) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:44.353077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:44.354551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:44.360841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 157/316] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 14:19:44 2022] GET /identity/v3/role_assignments?group.id=a1569170bb5f4439af64129064c70364&scope.project.id=4967f6c093674e39b531bed8fb1ec9b2&role.id=22d32c9fcd36484288e27a5f8630fa18 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:44.361701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.433108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.452106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 160/317] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:44.452106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.500265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079ee3a-a5d0-4848-b553-4baec46fec5c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.550326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:44.550964 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.551275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.551551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.552228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:44.552541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:44.553986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0efcf-fe1b-4b83-95ac-eac849711bc4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:44.563043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 158/318] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:44.564581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.564863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.565204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.588875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:44.590114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:44.614078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9c12-e227-4b2d-8473-7a54e3746aca None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.636199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.656464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:44.657197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.657482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.657757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.671851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 159/319] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:44.672794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.737098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.752769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:44.754169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:44.782770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2709c2-c19c-466f-bf85-a8667facc2cc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.821270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:44.821925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:44.822239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:44.822467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] PATH_INFO: `/v3/projects/4967f6c093674e39b531bed8fb1ec9b2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:44.823665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] RBAC: Authorizing `identity:get_project(project_id=4967f6c093674e39b531bed8fb1ec9b2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:44.825390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:44.827000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 160/320] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:44 2022] GET /identity/v3/projects/4967f6c093674e39b531bed8fb1ec9b2 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:44.827579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.873518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.876870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 161/321] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:44.882630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:44.961673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32913d-ef2d-49f6-bc04-8708c9bdd284 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:44.994243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:44.999724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.000329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.000607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.000915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.010680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb8758-7f1f-436d-bcbd-2f72f3a69803 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.047517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.048163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.048445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.048718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.050905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.053264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.054923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 162/322] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 14:19:44 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.055870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.081336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.082916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.122927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.169721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbca019-80ca-4bb3-a46b-9efd26fa37e8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.192376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 161/323] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:45.193314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.209936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.210574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.210852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.211131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.212269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.213819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.215380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 163/324] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.216160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.244388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.319902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e243624-c97c-421c-9b9f-ccb0d1b898f8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.331488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cebf27-acf1-4d8c-bce6-c2dbefe5a9bd None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.358802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.359438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.359712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.360076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.368433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.372169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.372772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.373075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.373175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:45.373343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.374129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] RBAC: Adding query filter params (group.id=a1569170bb5f4439af64129064c70364, role.id=95d84bb23f684a9da8785b860247f822, scope.project.id=4967f6c093674e39b531bed8fb1ec9b2) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:45.374426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.376006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:45.378067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:45.379508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 162/325] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:45.380637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.386645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 164/326] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/role_assignments?group.id=a1569170bb5f4439af64129064c70364&scope.project.id=4967f6c093674e39b531bed8fb1ec9b2&role.id=95d84bb23f684a9da8785b860247f822 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.387646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.513631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.516552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775241b-be6e-4325-9c37-b4264575da2a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.537820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7bd7e-18f5-4b9f-830f-594597271c49 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.565324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.566850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.567673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.568227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] PATH_INFO: `/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.570295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] RBAC: Authorizing `identity:get_project(project_id=20cddc5f50464f6fb97beba61f8a2e3b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.571824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.578367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 163/327] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/projects/20cddc5f50464f6fb97beba61f8a2e3b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.578367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.597127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.598063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.598474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.598843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.600125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.601661 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.603585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 165/328] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.604437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.675990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.731190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e8f80-db3a-4c8f-8859-1804a5c70814 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.752372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9652de3-fb85-463d-baa5-2cf4ba523741 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.786872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.787514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.787792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.788060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.788732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:45.789106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.790499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.798854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.799748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.800164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.800276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 164/329] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.801247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] PATH_INFO: `/v3/groups/a1569170bb5f4439af64129064c70364` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.801723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.802828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] RBAC: Authorizing `identity:get_group(group_id=a1569170bb5f4439af64129064c70364)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.804481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.806474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 166/330] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/groups/a1569170bb5f4439af64129064c70364 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.811272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.846673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.941171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1647ce-3a2a-4c87-891b-21823d45fb8d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.949454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd08767-5838-44e5-9a49-190d7eda709c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:45.973284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:45.984543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.985777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.985777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.985777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:45.987188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] RBAC: Adding query filter params (group.id=a1569170bb5f4439af64129064c70364, role.id=95d84bb23f684a9da8785b860247f822, scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:45.987188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:45.988885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:45.996549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 165/331] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/role_assignments?group.id=a1569170bb5f4439af64129064c70364&scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b&role.id=95d84bb23f684a9da8785b860247f822 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:45.997311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:45.997562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:45.998780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:45.999244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:45.999728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:46.092343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:46.093950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:46.123750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47864-691a-427f-ace6-3026f59671a3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:46.124872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:46.162553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:46.163192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:46.163637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:46.163966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:46.164721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] RBAC: Adding query filter params (group.id=a1569170bb5f4439af64129064c70364, role.id=22d32c9fcd36484288e27a5f8630fa18, scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:46.165088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:46.166715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:46.173145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 166/332] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/role_assignments?group.id=a1569170bb5f4439af64129064c70364&scope.project.id=20cddc5f50464f6fb97beba61f8a2e3b&role.id=22d32c9fcd36484288e27a5f8630fa18 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:46.241944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 167/333] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:19:46.247821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:46.333888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:46.387231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eec83e-36bb-4364-9d52-a9eb79139d41 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:46.427344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:46.427997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:46.428274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:46.428548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] PATH_INFO: `/v3/projects/e747be529418447d86ba59ccbdb4ada2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:46.429753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] RBAC: Authorizing `identity:get_project(project_id=e747be529418447d86ba59ccbdb4ada2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:46.431268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:46.432972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 167/334] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:19:46 2022] GET /identity/v3/projects/e747be529418447d86ba59ccbdb4ada2 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:46.444759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:46.555383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858463ed-261f-4539-b371-cbde22fe1654 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:46.579514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:46.592391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ba3f0-9c2b-46e4-8e23-fbe616a847f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:46.593044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ba3f0-9c2b-46e4-8e23-fbe616a847f0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:46.597021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ba3f0-9c2b-46e4-8e23-fbe616a847f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:46.597021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ba3f0-9c2b-46e4-8e23-fbe616a847f0 None admin] PATH_INFO: `/v3/groups/345178b7e9eb48f3b98eb7bd8a274719` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:46.597021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ba3f0-9c2b-46e4-8e23-fbe616a847f0 None admin] RBAC: Authorizing `identity:get_group(group_id=345178b7e9eb48f3b98eb7bd8a274719)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:46.597021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ba3f0-9c2b-46e4-8e23-fbe616a847f0 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:46.598325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 168/335] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 14:19:46 2022] GET /identity/v3/groups/345178b7e9eb48f3b98eb7bd8a274719 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:46.605472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:46.706175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254910c-7d3c-49d5-8ad5-10da22fdb64d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:46.730597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:46.741191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:46.741832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:46.742114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:46.742388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:46.747937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] RBAC: Adding query filter params (group.id=345178b7e9eb48f3b98eb7bd8a274719, role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=e747be529418447d86ba59ccbdb4ada2) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:46.747937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:46.747937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658c514-5ca0-44e2-babd-685ebf38ec54 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:46.751357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 168/336] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 14:19:46 2022] GET /identity/v3/role_assignments?group.id=345178b7e9eb48f3b98eb7bd8a274719&scope.project.id=e747be529418447d86ba59ccbdb4ada2&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:49.563350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2122a-b0e6-4ad9-9e17-09b166d81618 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:49.563895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2122a-b0e6-4ad9-9e17-09b166d81618 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:49.563895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2122a-b0e6-4ad9-9e17-09b166d81618 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:49.565254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 169/337] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:49.592227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194402f-7438-4054-b793-2e79a2e81052 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:49.593230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194402f-7438-4054-b793-2e79a2e81052 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:49.593701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194402f-7438-4054-b793-2e79a2e81052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:49.622291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f194402f-7438-4054-b793-2e79a2e81052 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:49.623879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f194402f-7438-4054-b793-2e79a2e81052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:49.727751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 169/338] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:49.828357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214dc7-a527-48dd-b1e2-d7d5edf3cc30 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:49.828957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214dc7-a527-48dd-b1e2-d7d5edf3cc30 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:49.829457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214dc7-a527-48dd-b1e2-d7d5edf3cc30 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:49.830993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 170/339] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:49.836765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86869b70-fc49-4f5a-b944-d6906176aa68 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:49.837340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86869b70-fc49-4f5a-b944-d6906176aa68 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:49.837821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86869b70-fc49-4f5a-b944-d6906176aa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:49.865917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-86869b70-fc49-4f5a-b944-d6906176aa68 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:49.867469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-86869b70-fc49-4f5a-b944-d6906176aa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:49.971244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 170/340] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:50.089880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eaf5d-2128-4f1a-86cf-2036c97215e8 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.090174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eaf5d-2128-4f1a-86cf-2036c97215e8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.090337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eaf5d-2128-4f1a-86cf-2036c97215e8 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.091992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 171/341] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:50.098157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.098445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.098726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.126152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:50.127370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:50.146176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eaf5d-2128-4f1a-86cf-2036c97215e8 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:50.216719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:50.259121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 171/342] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:50.316826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:50.354230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eaf5d-2128-4f1a-86cf-2036c97215e8 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:50.418362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:50.419209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.419493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.419766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.433790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:50.435605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:50.454242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:50.455254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:50.456194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:50.456194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:50.456194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:50.456194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 172/343] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:50.466285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:50.512157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff1f28-3144-46a8-94c9-6e4d6d12e731 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:50.564630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:50.566322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.566810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.567297 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.568812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:50.589149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:50.591374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:50.622939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:50.625468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:50.627059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:50.627059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:50.627059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:50.627059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 172/344] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:19:50.646873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:50.667663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263188f7-0ee7-42cc-bfcf-67e68aa62c3a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:50.701168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:50.732589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:50.733799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.734687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.735196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.736353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:50.736963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:50.739146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:50.762521 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 173/345] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:50.768231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:50.809946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c5ea7-310a-4bca-b202-31c0645e4722 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:50.853119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:50.854158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.854158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.854326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.868146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:50.869660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:50.885256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:50.886201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:50.887058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:50.887058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:50.887058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:50.887058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 173/346] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:50.887058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:50.926026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e296581-464e-43b5-89cd-7b10f5a608e8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:50.927850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:50.970267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:50.970879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.971170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.971432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:50.972119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:50.972919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:50.974639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b9cc0-677d-40ce-9277-d5d0cfb1ea69 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:50.989396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:50.994659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 174/347] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:50.995861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6aac-4b71-4e83-80dc-d97024535f6e None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:50.996173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6aac-4b71-4e83-80dc-d97024535f6e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:50.996445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6aac-4b71-4e83-80dc-d97024535f6e None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.001051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 175/348] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:51.001051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6aac-4b71-4e83-80dc-d97024535f6e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:51.017886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86da3f-80e9-422a-b8c6-632949d2f45a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:51.061334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed43db-8c5a-4e19-bfca-d9127b496711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:51.061925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed43db-8c5a-4e19-bfca-d9127b496711 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.062217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed43db-8c5a-4e19-bfca-d9127b496711 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.062473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed43db-8c5a-4e19-bfca-d9127b496711 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.063140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed43db-8c5a-4e19-bfca-d9127b496711 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:51.064676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed43db-8c5a-4e19-bfca-d9127b496711 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:51.125058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6aac-4b71-4e83-80dc-d97024535f6e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:51.161985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:51.162513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.162787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.163065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.163700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:51.163995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:51.165509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:51.180694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 176/349] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:51.181891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:51.183550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 174/350] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:50 2022] POST /identity/v3/users => generated 246 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:51.185150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e30cd-9514-42a3-b553-f98b2d8cdf0f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.185507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e30cd-9514-42a3-b553-f98b2d8cdf0f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.185858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e30cd-9514-42a3-b553-f98b2d8cdf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.211322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-516e30cd-9514-42a3-b553-f98b2d8cdf0f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:51.212712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-516e30cd-9514-42a3-b553-f98b2d8cdf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:51.217890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:51.305939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 175/351] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:51.309440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6536-3715-4b89-9317-ea91ebd9ae95 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.309805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6536-3715-4b89-9317-ea91ebd9ae95 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.310234 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6536-3715-4b89-9317-ea91ebd9ae95 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.311969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 176/352] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:51.312629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad6536-3715-4b89-9317-ea91ebd9ae95 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:51.351140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef6c6d-2c52-46cb-8599-f945827d1908 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:51.403717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dded446-3ef4-45a2-9010-f0899b3ca991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:51.404973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded446-3ef4-45a2-9010-f0899b3ca991 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.405708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded446-3ef4-45a2-9010-f0899b3ca991 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.406173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded446-3ef4-45a2-9010-f0899b3ca991 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.407512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dded446-3ef4-45a2-9010-f0899b3ca991 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:51.411040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dded446-3ef4-45a2-9010-f0899b3ca991 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:51.414824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:51.459100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad6536-3715-4b89-9317-ea91ebd9ae95 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:51.497421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:51.498086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.498355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.498656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.499388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:51.500789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:51.521213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 177/353] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:51 2022] POST /identity/v3/users => generated 248 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:51.523176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.523679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.524268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.550036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:51.552423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:51.629194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 177/354] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:51 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:51.631033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:51.656679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 178/355] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:51.817734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652289-330b-4012-b721-1ca7c03c63fa None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:51.853473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:51.866575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:51.867243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:51.867522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:51.867793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:51.880421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:51.881996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:51.899296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:51.905454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:51.906384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:51.906384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:51.906384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:51.906384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 178/356] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:51.909830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:51.922911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:51.994490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:52.050953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ab4f3-026b-4026-8ce4-056a7256c8d9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:52.100198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:52.101142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:52.101844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:52.102256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:52.104312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:52.104768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:52.106466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:52.123934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 179/357] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:19:52.131447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:52.152371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9136f-1873-47ea-bd38-0b271e5ebc72 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:52.208888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:52.215464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:52.216083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:52.216360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:52.219227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:52.219227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:52.219227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:52.257695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211aef7-0a90-4b73-98af-3450eba06c3c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:52.274040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.sql.core [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 14:19:52.274040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 14:19:52.274040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [parameters: {'id': 'c7fab87aaa884bf5a4e36dc3fe8d9275', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 14:19:52.274040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81802) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result.read() Nov 02 14:19:52.281798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:19:52.282741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application yield resource Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise self.value Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application session.commit() Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 14:19:52.283654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.session.flush() Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application compat.raise_( Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 14:19:52.285228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:19:52.286087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result.read() Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:19:52.286877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [parameters: {'id': 'c7fab87aaa884bf5a4e36dc3fe8d9275', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:52.287649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 14:19:52.290250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:52.292399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 179/358] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 14:19:51 2022] POST /identity/v3/roles => generated 144 bytes in 290 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:52.292399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:52.336355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:52.337701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:52.337987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:52.338258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:52.340167 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:52.343512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:52.369232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:52.459719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af77363-5c50-4409-94e4-63d64617c1cd None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:52.496544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 180/359] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:52 2022] POST /identity/v3/users => generated 249 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:52.513832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ad9e1-4973-4ea5-8a59-ac3c3fbedbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:52.514514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad9e1-4973-4ea5-8a59-ac3c3fbedbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:52.514788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad9e1-4973-4ea5-8a59-ac3c3fbedbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:52.515064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad9e1-4973-4ea5-8a59-ac3c3fbedbb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:52.533163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ad9e1-4973-4ea5-8a59-ac3c3fbedbb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bb9ad9e1-4973-4ea5-8a59-ac3c3fbedbb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:52.540896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:52.542704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:52.544380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 180/360] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:19:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:52.553038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:52.829779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cabb8-7e80-4a75-b177-207bd9a2d4d3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:52.881542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:52.919905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:52.920542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:52.920823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:52.921131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:52.921818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:52.922123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:52.923762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651821ee-3806-4d9d-80ef-1385ed0951cb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:52.945263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 181/361] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:53.897888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77b09-62dc-4b4c-8a0e-e2d77c1e4cbc None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:53.898889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77b09-62dc-4b4c-8a0e-e2d77c1e4cbc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:53.899380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77b09-62dc-4b4c-8a0e-e2d77c1e4cbc None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:53.907754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 181/362] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:53.912394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456973e0-3d1d-4801-a080-9b3177d46d9a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:53.912950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456973e0-3d1d-4801-a080-9b3177d46d9a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:53.913312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456973e0-3d1d-4801-a080-9b3177d46d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:53.943954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-456973e0-3d1d-4801-a080-9b3177d46d9a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:53.945185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-456973e0-3d1d-4801-a080-9b3177d46d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:54.055732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 182/363] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:54.435690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f77b09-62dc-4b4c-8a0e-e2d77c1e4cbc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:54.588031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:54.593771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f77b09-62dc-4b4c-8a0e-e2d77c1e4cbc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:54.604578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f5f45-6bf1-462f-beb3-74fc99bbe934 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:54.605098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f5f45-6bf1-462f-beb3-74fc99bbe934 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:54.605585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f5f45-6bf1-462f-beb3-74fc99bbe934 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:54.608520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 183/364] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:54.616124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61152e91-0465-4b4a-ad6f-040b40840c0c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:54.616599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61152e91-0465-4b4a-ad6f-040b40840c0c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:54.617136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61152e91-0465-4b4a-ad6f-040b40840c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:54.649058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-61152e91-0465-4b4a-ad6f-040b40840c0c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:54.651253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-61152e91-0465-4b4a-ad6f-040b40840c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:54.701533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:54.702734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:54.703200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:54.703677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:54.738917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:54.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:54.757409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:54.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:54.760379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 182/365] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:19:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:54.767758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:54.826102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 184/366] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:54.827371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e158bc-6606-44b2-bc74-1f53f2f5edad None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:54.827652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e158bc-6606-44b2-bc74-1f53f2f5edad None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:54.827920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e158bc-6606-44b2-bc74-1f53f2f5edad None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:54.833171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 185/367] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:54.835244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:54.835585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:54.835921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:54.881656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:54.882632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:54.941376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcbdd7-3c93-4081-bd40-b53906c5745c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:54.974636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:55.002241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 186/368] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Nov 02 14:19:55.029794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.030424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.030695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.030960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.031639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:55.031936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:55.033458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:55.048546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 183/369] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:19:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:55.054626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.182414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.215126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:55.230581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c62f5-a289-43e3-a19c-19d2af932d3b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:55.304027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.305040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.305545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.306048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.338464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:55.341680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:55.374232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:55.375516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:55.377547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:55.377547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:55.377547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:55.377547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 187/370] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:55.378462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.387709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7f2e9-98fd-4b29-8684-a058604bd377 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:55.421803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:55.438361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.439961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.440517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.441752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.458153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:55.465487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:55.466433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:55.467299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:55.470292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 184/371] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:55.471335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.515945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a15a3-7de3-4c81-bc74-187789bbe3eb None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:55.558364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.559449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.559870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.560614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.575356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:55.582856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:55.584354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:55.585385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:55.587807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 188/372] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:55.588892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:55.589056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.616802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ce69-c9a5-4ed0-98c8-7764dc3b4460 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:55.662766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.663692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.664316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.664587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.665509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:55.666071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:55.667827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:55.683679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 185/373] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:55.685164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.730279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:55.753384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f2e76-af54-49c2-8c91-d338c80186f9 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:55.808757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.809288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.809764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.810363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.811258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:55.811918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:55.813597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:55.821430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c641d-9dca-4bcb-beba-0b02bc47493e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:55.823763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 189/374] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:55.829216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.864080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:55.864817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.865124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.865467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.866315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:55.866620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:55.868094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160051ae-de23-4b94-9875-0bd56103feff None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:55.877845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 186/375] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:55.879667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df180-3f51-4f0b-b5e3-f123531f80a3 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:55.879953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df180-3f51-4f0b-b5e3-f123531f80a3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:55.880313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df180-3f51-4f0b-b5e3-f123531f80a3 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:55.882008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 187/376] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:55.882744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df180-3f51-4f0b-b5e3-f123531f80a3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:55.889883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:55.974742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c873ee-20d2-4e1f-9e48-630fa22a4b73 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.019977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.020703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.020988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.021353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.030413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.032007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.033511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df180-3f51-4f0b-b5e3-f123531f80a3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:19:56.045964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:56.047166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 190/377] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:56.048332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.074873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:56.089242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.089878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.090155 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.090434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.101901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.103640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2291c0d6-7692-4a17-841c-fff7494f1b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:56.118861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:56.120061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:56.120918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:56.120918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:56.120918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:56.120918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 188/378] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:56.121262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.121538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.121772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.140547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:56.141779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:56.197590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:56.202084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c32e7c-1025-4d78-ac88-91a1dd8650a2 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.214579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 189/379] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:56.215273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.243440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.244613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.244613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.244613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.257725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.259179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-7485535f-31d1-46f2-ae71-94ae6c93ad98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:56.274553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:56.276214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:56.277081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:56.277081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:56.277081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:56.277081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 191/380] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:56.277778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33d1e-666d-42ca-bb6f-507fea709b77 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.277887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33d1e-666d-42ca-bb6f-507fea709b77 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.278113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33d1e-666d-42ca-bb6f-507fea709b77 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.281215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 192/381] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:19:56.281215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33d1e-666d-42ca-bb6f-507fea709b77 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.359294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad5ed-18d5-443f-901e-6d89f0064593 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.368713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:56.405869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.406607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.406885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.407156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.408117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:56.408871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.411055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.419125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33d1e-666d-42ca-bb6f-507fea709b77 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.461790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.462449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.462721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.462996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.463695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:56.464000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.465514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7346ab-060e-495d-95a9-1f3acd2798dc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.468939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 190/382] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:56.470413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.493379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 193/383] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:56.493379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.493379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.493379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.514381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:19:56.515656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:19:56.531129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:56.599832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4f31-33c7-4667-810d-eb81f7ab4d32 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.607991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 194/384] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:19:56.608896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.649660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.650290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.650568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.650838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.651642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:56.651945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.653467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.678170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 191/385] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:56.678980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.692981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:56.755925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce326674-b02a-47f1-89d6-fd39950c21a5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.815074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.816245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.816583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.816766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.830134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.833198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.840450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126a8b3-1fd8-48d7-810a-d22a0ca392e0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.843132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:56.856281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:56.857468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:56.858328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:56.858328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:56.858328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:56.858328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 195/386] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:56.859365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.897842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:56.898661 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:56.899334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:56.899603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:56.909199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:56.910875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:19:56.926079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:56.927192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 192/387] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:56.928275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:56.990555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5aada-82f9-455e-bc1c-11101b276356 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:56.993008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.034314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.035279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.035605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.035977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.053503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:19:57.054482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26dad-6d65-4663-8931-1b9ba45dd00a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:19:57.061548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:57.062637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:57.063560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 196/388] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:57.066274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.104178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.104802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.105102 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.105401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.113593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.115057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:19:57.126483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:57.127624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 193/389] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:19:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:57.129566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.167553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.193691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c3347-3df7-42d9-9225-d08079c7fe00 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.242397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.244373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.245069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.245729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.247004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:57.247602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.249527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.263918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 197/390] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:57.264856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.287229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9214-1a8e-4d8a-aba3-27167cb7e088 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.305979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.333027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.333681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.333950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.334355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.350678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.352255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:57.372856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:57.374829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:57.376419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:57.376419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:57.376419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:57.380280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 194/391] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:57.381226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.405449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf2b1-3a1a-4aa2-b1d0-3a762776dce2 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.446092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.446654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.446927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.447226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.447949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:57.448248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.449730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.461467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.485636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865a9af-9274-4a3b-ad0b-f8541b63a4e5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.490077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 198/392] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:57.490827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.522796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.523395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.523785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.524043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.524706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:57.525018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.526395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.535823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 195/393] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:57.536716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.583016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.605199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe9bfb-c641-4434-837c-4305dc55fb1a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.644941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.645603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.646543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.646543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.646988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:57.647287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.648773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.666948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6161ecd-994e-480d-872f-506b048d2eff None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.669855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 199/394] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:57.671219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.706739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.707298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.707575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.707837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.715239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.716696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.718612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:19:57.731469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:57.735684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 196/395] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:57.737676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.827230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3f821-840b-45b8-96eb-1daa30bd880c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.881609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.882531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.882903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.883349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.884431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:57.884829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.889607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.895849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd75df-17de-4c00-b3dc-d9bdd1ecda9e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:57.904644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:57.910600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 200/396] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:57.915164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:57.963512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:57.964065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:57.964339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:57.964621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:57.979768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:57.984205 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:57.999893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:58.002018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:58.003692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:58.003692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:58.003692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:58.003692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 197/397] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:58.003692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.062953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:58.075014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d27f8-1b66-4cda-a95b-873a9612b7a9 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.118106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.118719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.118991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.119255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.119830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac37f3-73d3-459d-8aa8-272ad4c6c2af None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.131970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.133485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:58.149602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:58.150649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:58.151584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:58.151584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:58.151584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:58.151584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 201/398] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:58.151878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.162185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.162816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.163091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.163354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.174741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.176337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:58.191777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:58.192873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:58.194171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:58.194171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:58.194171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:58.194171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 198/399] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:58.194171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.217680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:58.277699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333710c0-bc5f-4c8f-92ce-75fa3d294d24 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.318030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78349a0-cbc3-4ab3-be49-32315d4b86d1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.318265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.318885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.319156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.319426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.319955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:58.320107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:58.320413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.321922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.341140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 202/400] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:58.341993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.360861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.362612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.362612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.362612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.362926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.364278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.450686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 199/401] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:19:58 2022] POST /identity/v3/users => generated 247 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:19:58.451526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.457901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eb12f-dd4b-4f9c-a465-c8e0c791e009 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.478648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:58.501195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.501864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.502147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.502420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.504424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:58.504674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.506093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.527429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 203/402] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:58.528832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.587242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69a04d-31cf-491d-a730-fcd07a434d65 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.631919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.632578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.632825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.633131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.633859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:58.634216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.635596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.650054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 200/403] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:58.651000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.661999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef402f-9f30-4cd7-95b5-0220ab06b28a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.675256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:58.707539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.708162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.708430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.708694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.709374 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:58.709675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.711066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.724301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 204/404] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:58.724890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.763247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:58.781939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046608-b25c-4e7a-be62-6001cf87d4a2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.827770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.828556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.828907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.829402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/ea17166a0ace485dbf6cadaae7731ed6/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.844837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737440fb-412d-40e5-8062-4f3a5c68e65d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:58.849118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=ea17166a0ace485dbf6cadaae7731ed6, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.874813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.883276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:58.883970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:58.884254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:58.884523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:58.892165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:58.894281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:19:58.909622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:58.910982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 205/405] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:58.916897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:58.966393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 201/406] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:58 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/ea17166a0ace485dbf6cadaae7731ed6/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:58.974210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.040103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:59.058526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397652f-a625-4108-bc88-032cc8a89f79 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.085126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:59.122700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.124223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.124801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.125300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.136124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.136403 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640aad8a-24fa-4543-ae5f-bdde0d0b0978 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.138180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:19:59.155013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:59.156061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 14:19:59.157014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:59.160041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 206/407] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:59.161317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.183200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.184003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.184285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.184549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.196070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.197571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:19:59.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:59.213261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 202/408] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:59.215397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.348678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56d00-6e96-4a96-9d55-4551ed34d40a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.392252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.393379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.394370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.394936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.396324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:59.397191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.398873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.401864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c990-d847-4333-8703-ea035b3c7196 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.413473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 207/409] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:59.419030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.436710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:59.459560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:59.474036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.474637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.475047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.475453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.476302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:59.476709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.479420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.511347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 203/410] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Nov 2 14:19:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:59.512867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.552433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fca3d4-5265-479e-b6e6-eac97572e66b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.597335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.598536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.599164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.599575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.600448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:19:59.601219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.604125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.625109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 208/411] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:19:59.626123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.657391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327a21e-cb63-4226-a7e5-f477f77d9d40 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.698591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.699615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.700091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.700556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/20e8f2a80ea142b7acb78ea661b0e47d/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.717805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=20e8f2a80ea142b7acb78ea661b0e47d, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.719713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.726088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:19:59.753945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f35bd-303c-44e0-967f-0b78ce9f0ded None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:19:59.792975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:19:59.793631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:19:59.793910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:19:59.794175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:19:59.807480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:19:59.808944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:19:59.823721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 204/412] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:59 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/20e8f2a80ea142b7acb78ea661b0e47d/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:19:59.825844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:19:59.844045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:19:59.845366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:19:59.850906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:19:59.850906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:19:59.850906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:19:59.850906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 209/413] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:19:59.857507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:19:59.890210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:00.006362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e896dd-e7c0-4027-983d-1e86dd47033f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:00.066326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:00.067550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:00.068168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:00.068652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/287fbe09913b454c9fcd58febbca6b30/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:00.082404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ee35a-c7c1-4827-91aa-ab9e6cc2ab1b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:00.087875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=287fbe09913b454c9fcd58febbca6b30, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:00.090308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:00.114033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:00.128814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:00.130618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:00.131135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:00.131641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:00.132549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:00.133120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:00.134816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:00.153869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 210/414] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:00.157821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:00.237262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 205/415] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:19:59 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/287fbe09913b454c9fcd58febbca6b30/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:00.344483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294cad5-0853-4c72-9875-60b3e51d6bc6 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:00.400536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:00.401135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:00.401464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:00.401742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:00.414112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:00.415669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:00.434798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:00.435764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:00.437067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 211/416] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:00.445038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:00.573415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:00.593652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41f87-d520-4697-85dd-ca8998e17c86 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:00.645471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:00.646091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:00.646367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:00.646637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:00.647406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:00.647712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:00.649197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fd888-5adb-4a91-90fb-e6d2230310da None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:00.680847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 206/417] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:00.687002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:00.763443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:00.844531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdae74-cd1d-48fe-a007-5184bfeac331 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:00.893143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371123c9-1392-4bf3-be8c-574bad16bb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:00.894327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371123c9-1392-4bf3-be8c-574bad16bb03 None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:00.894881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371123c9-1392-4bf3-be8c-574bad16bb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:00.903185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371123c9-1392-4bf3-be8c-574bad16bb03 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/b981e25e0b8d46f298b05b261433de94/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:00.925526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371123c9-1392-4bf3-be8c-574bad16bb03 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=b981e25e0b8d46f298b05b261433de94, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:00.927152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371123c9-1392-4bf3-be8c-574bad16bb03 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:01.034872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 212/418] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:00 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/b981e25e0b8d46f298b05b261433de94/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:01.105295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045d523-5730-4945-ab20-6372698a1103 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:01.105829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045d523-5730-4945-ab20-6372698a1103 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:01.105914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045d523-5730-4945-ab20-6372698a1103 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:01.107328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 207/419] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:01.113723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca5d8f-ad2a-4d6d-bc3e-654b723721e8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:01.114027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca5d8f-ad2a-4d6d-bc3e-654b723721e8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:01.114270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca5d8f-ad2a-4d6d-bc3e-654b723721e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:01.139827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5cca5d8f-ad2a-4d6d-bc3e-654b723721e8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:01.141298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5cca5d8f-ad2a-4d6d-bc3e-654b723721e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:01.249164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 213/420] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:20:01.535971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045d523-5730-4945-ab20-6372698a1103 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:01.548208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbaa05-7fdc-4134-9974-e56d9421de06 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:01.548848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbaa05-7fdc-4134-9974-e56d9421de06 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:01.549377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbaa05-7fdc-4134-9974-e56d9421de06 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:01.551486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 214/421] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:01.556987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:01.557599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:01.558104 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:01.594659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:01.597851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:01.641448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:01.686019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 215/422] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:20:01.762392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045d523-5730-4945-ab20-6372698a1103 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:01.815137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:01.816233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:01.816506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:01.816774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:01.834326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:01.840962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:01.841923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:01.842710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:01.845478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 208/423] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:01.852518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:01.967958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ddd07-3d60-46f8-8f79-e8cf0147b6ba None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.010909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.011928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.012401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.012872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.013796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:02.014418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:02.016075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:02.026361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 216/424] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:02.032594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:02.043764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:02.090950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:02.113696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:02.199512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae3395-6824-4ebf-a192-ad147e298913 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.240527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.241205 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.241485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.241757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.254230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:02.255986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:02.279175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:02.280138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:02.280982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:02.280982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:02.280982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:02.280982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 209/425] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:02.289085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:02.304820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbcbe-e37b-4297-960d-7a05d400fb1b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.342428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.343075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.343353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.343667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.362897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:02.370050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:02.370991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:02.371784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:02.372389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 217/426] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:02.379716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:02.433398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:02.520511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56934b9-5410-4ad4-b07e-fd19a0d3f15d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.555533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:02.605992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.606616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.606895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.607161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.607832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:02.608129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:02.609566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:02.614561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263d901-cd9e-4078-af89-f9ed2a1a8573 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.644561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 210/427] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:02.649617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:02.683603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.684590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.684873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.685237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.686017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] RBAC: Adding query filter params (name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:02.686425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:02.687801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c157-52e1-4ab6-a872-2b1faa38f51f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:02.699191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 218/428] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:02.701249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620705a-3b41-4a7e-bdd6-651ed75de69d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.701536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620705a-3b41-4a7e-bdd6-651ed75de69d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.701815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620705a-3b41-4a7e-bdd6-651ed75de69d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.703288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 219/429] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:02.705125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620705a-3b41-4a7e-bdd6-651ed75de69d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:02.797138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c0e29-0b95-4d37-854f-0e3856d0f0a9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.822882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:02.845976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.847138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.847416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.847686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.850475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620705a-3b41-4a7e-bdd6-651ed75de69d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:02.857123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:02.858141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-38d3694e-9c0b-4a62-a1c6-c72ab3310605 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:02.871687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:02.872934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 211/430] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.879206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.879876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.923544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:02.925773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:02.955297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:02.956669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:02.957547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:02.958338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:02.967905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:02.974441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:02.984287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:02.997387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:02.998463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:02.999498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:02.999498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:02.999498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:03.004795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 220/431] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:03.006652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.030749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 212/432] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:03.031917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.140711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5694-94d1-44d0-8789-f8a2a04fd20c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.163911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0042f-21fd-4efe-9e75-eccd66a3dacf None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.177733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.180093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.180907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.181393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.181831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.182727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:03.183227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.185032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562f47c-d530-4e5b-91b8-7a4dfb631ddb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.205361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 221/433] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:03.207526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6160e7-4657-4b48-9f1b-669640cb1002 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.208049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6160e7-4657-4b48-9f1b-669640cb1002 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.208544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6160e7-4657-4b48-9f1b-669640cb1002 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.208862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247349ed-5942-4268-aad5-65517213adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.209530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247349ed-5942-4268-aad5-65517213adbf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.209810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247349ed-5942-4268-aad5-65517213adbf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.210082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247349ed-5942-4268-aad5-65517213adbf None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.210738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247349ed-5942-4268-aad5-65517213adbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:03.211040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247349ed-5942-4268-aad5-65517213adbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.211687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 222/434] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:03.212418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247349ed-5942-4268-aad5-65517213adbf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.212800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6160e7-4657-4b48-9f1b-669640cb1002 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.233209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 213/435] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:03.235159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.235453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.235731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.261385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:03.262985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:03.281176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.342351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 214/436] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:03.343607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.353049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6160e7-4657-4b48-9f1b-669640cb1002 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.388980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.389644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.389920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.390189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.401688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.403131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:03.421260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:03.422225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:03.423073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:03.423073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:03.423073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:03.423073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 223/437] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:03.424071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.466587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.470392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef73073-305f-419d-896d-c5a8d3691f8a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.508739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.509641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.510081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.510517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.519395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.521299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:03.533596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:03.535371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:03.536497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:03.538924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 215/438] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:03.539917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.554178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.558116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4453d2-1637-46c0-bb28-79182476cae1 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.600479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.601368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.601659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.601938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.615548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:03.620946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:03.621867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:03.622762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 224/439] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:03.623421 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.659844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ae4c2-6b7c-41a8-86c2-a6d3499e46ef None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.697222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.697832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.698109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.698522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.699078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:03.699425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.700805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.714598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 216/440] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:03.715457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.736235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.740991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386144cc-bbf0-4a32-8816-d07c8c469a27 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.786857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.787485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.787757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.788026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.788772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:03.789107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.790636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.812045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 225/441] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:03.812781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.834909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec2603-ba70-4499-9659-fe8d5676f2b9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.835327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.873797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.874453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.874686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.874959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.875631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:03.875930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:03.877379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:03.886750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 217/442] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:03.887594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:03.930339 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a923-4d11-4f5e-8dd8-3add31a95bcb None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:03.967193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:03.967818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:03.968092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:03.968357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:03.977147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:03.980080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:03.984970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:03.993239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 226/443] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:03.994235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.001236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654ac95-85b3-45b6-b0ec-9c8e7a672d73 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.040254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.040866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.041172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.041446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.048685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.050864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3c008c28-12de-4529-bb37-14f7be44a507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:04.062318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:04.063362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:04.064386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:04.068179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 218/444] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:04.069715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884e115-b839-4e02-819f-0be843de01e6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.069996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884e115-b839-4e02-819f-0be843de01e6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.070269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884e115-b839-4e02-819f-0be843de01e6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.071601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 219/445] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:04.072286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884e115-b839-4e02-819f-0be843de01e6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.074798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.098836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8f5b-2de4-48f0-8df3-e8c07f819a3d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.137885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.138507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.139355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.139355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.186787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884e115-b839-4e02-819f-0be843de01e6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.212634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.214143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.224516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.225191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.225471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.225740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.235358 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.238349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.239866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:04.254939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:04.256062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:04.256912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:04.256912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:04.256912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:04.256912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 220/446] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:04.257161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.298149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.321333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 227/447] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:20:04.322271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.367701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714112f-7243-4f93-a53b-ef704edce51b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.402729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.403379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.403656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.403930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.404590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:04.404889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.406305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b570938-932c-42ff-81c0-6214de7ae202 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.415072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 221/448] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:04.416294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.416867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.416867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.427516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24349-0fd3-4472-9b9b-b8c09f288c7a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.435634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:04.436877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:04.466278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.466919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.467201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.467472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.468165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:04.468471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.469929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.484221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 228/449] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:04.485106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.494585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.527502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 222/450] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:04.528372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.530917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.619832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b6421-8ec9-4854-b260-41d7aa1828c3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.646256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878c3fe-5921-4f70-882a-2814883dd125 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.658765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.659668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.660036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.660403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.661393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:04.662211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.663761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.677398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 229/451] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:04.678120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.689094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.689688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.689992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.690859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.702369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.703854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:04.720018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:04.721436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:04.723129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:04.723129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:04.723129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:04.725893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 223/452] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:04.726823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.768788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.797278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:04.803227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adb87a-ead3-4204-a174-fe62de8a80ac None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.840584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.841246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.841522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.841792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.841933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779440ae-6cef-4438-9e83-ed3ecd2a52d8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:04.853491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.855080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:04.872835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:04.873963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:04.874827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:04.874827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:04.874827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:04.874827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 230/453] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:04.874827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:04.883509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:04.884672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:04.884947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:04.885245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/0435bede480b4f83aee82223ef80836f/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:04.902546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=0435bede480b4f83aee82223ef80836f, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:04.904022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:04.982039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076b06-9ac8-40c8-ab9f-fc24921cb7b7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.019029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 224/454] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:04 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/0435bede480b4f83aee82223ef80836f/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:05.033139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.033378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.033378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.033378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.033378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.045039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.045039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:05.050221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:05.051651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 231/455] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:05.052689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.071918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.133689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.152101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20760d-bf26-4ead-b5cf-a21b6f58f19f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.187553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da68ca-d190-4104-aad1-0942efaf3a0d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.193813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.194447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.194744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.194986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.195648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:05.195953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.197420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.215997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 225/456] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:05.215997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.236536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.237777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.238043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.238308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.252366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:05.260325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:05.261425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 232/457] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:05.262928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.344893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673393f5-0946-44ee-8967-3febba7fcf78 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.354332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.384717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.385354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.385634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.385905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.386606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:05.386907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.388318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.391151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf646-6c2e-4e10-8aa5-c27967a09758 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.403809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 226/458] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:05.404947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.415638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.438794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.439445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.439771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.440568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.441420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:05.441725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.443148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.463439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 233/459] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:05.464798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.519398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be83a-55c4-4f97-8633-da8ef18ff002 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.560091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.560729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.561048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.561299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.569040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.570628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:05.584421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:05.590820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 227/460] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:05.591894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.598409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647decc6-4165-4fd3-a485-fe4ab76967a2 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.631078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.639618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.640232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.640507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.640769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.641678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] RBAC: Adding query filter params (name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:05.645350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.645350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.658130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 234/461] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:05.658130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.665471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.702980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a014d-5f32-4b22-80c0-d2beb34c5f0a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.743001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.743659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.743940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.744210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.753990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.755463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.765475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d5dec-faab-4b44-bc4a-8af07f59d117 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:05.768725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:05.769874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 228/462] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:05.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:05.809832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.810440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.810728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.810982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.876870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.885724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c63f-27f9-4213-894e-24c5eac18c26 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:05.892194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.894798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.915804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:05.925072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:05.925706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:05.926103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:05.926373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:05.927072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:05.927506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:05.928994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:05.947929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 229/463] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:05.947929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.027681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 235/464] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:20:06.027681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.068496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7f897-ce64-449b-82a9-113f411c5214 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.106686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.107345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.107628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.107903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.119364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.120852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.130360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:06.138528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:06.150150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:06.151017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:06.151017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:06.151017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:06.151017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 230/465] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:06.151017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.151285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec13ab9-3723-4b31-ab68-6fbcc5756c50 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.183817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.184427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.184702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.184974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.185734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:06.186041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.187576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.189640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.208375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 236/466] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:06.210434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.266084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08245c68-8cbc-4328-a044-2c5c9e3f0656 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.307428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.307959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.308228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.308494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.309252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=ea17166a0ace485dbf6cadaae7731ed6) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:06.309556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.310986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.318410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 231/467] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/role_assignments?user.id=ea17166a0ace485dbf6cadaae7731ed6&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:06.319224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.335601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0556-bbef-4aca-83e7-8914e042d965 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.383040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.384018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.384475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.384945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.401415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.403018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.415828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:06.423812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:06.425679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:06.426580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:06.426580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:06.426580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:06.426580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 237/468] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:06.429222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.494972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce567bd7-2b51-46ab-937f-c0c9c4d96478 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.497983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.548911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.550579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.550896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.551171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.551825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:06.552125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.553734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.569368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 232/469] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:06.570175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.571523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9097e5-c1d3-42ba-be4a-06c09b017333 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.616321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.616970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.617309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.617650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.691690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc37c1-738b-47b0-9a88-97e45a952f0d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.703371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.706919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.709325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.732746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.733397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.733741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.734020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.735179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:06.735484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.738087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.762287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 233/470] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:06.766208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.796373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.848672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 238/471] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:20:06.850162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.885432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7752f-89fb-42ea-87ca-fecf40f5054c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:06.935598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:06.937172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:06.937737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:06.938199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:06.950055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:06.952134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:06.970191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:06.971717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 14:20:06.972685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:06.980165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 234/472] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:06.980165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:06.991736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:06.995490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcbab7-063e-41a1-866a-f53c6fa62638 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.037618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.038741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.039226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.039698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.048202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.050805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:07.065409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:07.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:07.067590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:07.069828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 239/473] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:07.070831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.076744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:07.097894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e1ada-7c66-4b49-9818-ea525c140d79 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.146749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.147948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.148371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.148803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.163676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.165475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:07.182879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:07.184093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:07.185178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:07.185178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:07.185178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:07.186561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 235/474] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:07.187525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.216971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9793-1f12-4468-9aaf-31397e7760fe None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.261692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.262568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.263010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.263437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.264322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:07.264951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.267120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.292640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 240/475] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:07.294116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.304478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b595ea-0d2f-435d-8df7-b22f4b2615f0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.317604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:07.340505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.341143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.341416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.341679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.342375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:07.342672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.344115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.345994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:07.364072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 236/476] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:07.365089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.416315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39c307-ae6b-47a0-9f0c-dc83b69ae0ee None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.455964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.456672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.456953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.457279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.458347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:07.458654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.460393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd549337-f3db-4adf-b34a-2d9f3c4fadea None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.472231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 241/477] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:07.473509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc130eb-f545-4246-8118-51f9bf74fc2f None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.473790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc130eb-f545-4246-8118-51f9bf74fc2f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.474061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc130eb-f545-4246-8118-51f9bf74fc2f None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.479719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 242/478] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:07.479719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc130eb-f545-4246-8118-51f9bf74fc2f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.492633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd16d9-ce56-43eb-b43e-272692b04677 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.530885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3e006-417a-4cc8-a21e-2b8d0b183fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.531559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3e006-417a-4cc8-a21e-2b8d0b183fda None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.531742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3e006-417a-4cc8-a21e-2b8d0b183fda None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.532010 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3e006-417a-4cc8-a21e-2b8d0b183fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.571802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:07.599785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc130eb-f545-4246-8118-51f9bf74fc2f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:07.619450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3e006-417a-4cc8-a21e-2b8d0b183fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.621508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3e006-417a-4cc8-a21e-2b8d0b183fda None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.646243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.646854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.647138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.647399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.648125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=20e8f2a80ea142b7acb78ea661b0e47d) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:07.648430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.649988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.656599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 243/479] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/role_assignments?user.id=20e8f2a80ea142b7acb78ea661b0e47d&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:07.657254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.782579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 237/480] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 14:20:07.791665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.792004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.792254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.792663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474b0b5-e947-42ae-9843-0b4becee8dff None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:07.813824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:07.815738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:07.842350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:07.843643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:07.844141 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:07.844804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:07.860895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:07.865194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:07.880710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:07.881790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:07.882749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:07.884424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 244/481] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:07.890773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.894864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:07.911288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 238/482] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:07.917207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:07.944649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:08.107062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c9efc-d0f0-4512-9110-7ae40cbbaa8f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.123167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02558fe-8bdf-4a42-a64c-6a38bd0b0efb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.181535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.182188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.182464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.182744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.201058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.201359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.201621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.202139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.204377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:08.205601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.207176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.210063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.211578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.224159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 239/483] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:08.225649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:08.228815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:08.229915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:08.230874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:08.230874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:08.230874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:08.230874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 245/484] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:08.231496 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.302083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:08.338911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:08.359952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f5c6-9d05-4974-a150-e99ce2eb7bc5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.365037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40051b6-7c7e-4774-bd2e-4ef6f6a7a414 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.407211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.407886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.408173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.408447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.411991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.412624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.412904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.413211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.413961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=287fbe09913b454c9fcd58febbca6b30) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:08.414272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.415842 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.422880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:08.423850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 246/485] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/role_assignments?user.id=287fbe09913b454c9fcd58febbca6b30&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:08.426224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:08.435267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:08.436244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:08.437072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:08.441243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 240/486] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:08.447384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.666480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4900cc-9ddf-44d7-8e99-b2857f33c96c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.677163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ae9d9-b9e3-4ee8-a248-0732aacfbdd9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.686949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:08.706569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.707183 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.707460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.707723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.710091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:08.710091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.710427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.722533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.723215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.723496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.723767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.724443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:08.724744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.726288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.727655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 247/487] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:08.734045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.743230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 241/488] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:08.749040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.760156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:08.855556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a2cbf-7007-4ca3-bdca-a45bebfc2af8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.874675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66328e18-325c-4dc4-b1e9-6636d6516fc3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:08.894033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.894744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.895025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.899903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.903315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.904092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.917213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:08.917842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:08.917963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:08.918741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:08.919572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:08.919970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:08.919970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:08.920088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 248/489] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:08.926543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.938744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:08.940421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:08.958275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:08.959812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:08.960747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:08.960747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:08.960747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:08.960747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 242/490] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:08.966563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:08.976090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:09.046445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:09.050109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90e9f-ce25-4b1a-a0dd-ea44401601a4 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:09.098000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:09.098636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.098911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.099178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.099951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:09.100162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:09.101692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4842-12f8-4762-aa1e-eb84704c80ef None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:09.103779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09646013-2217-4612-980b-1808e1ac7d35 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:09.115726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 249/491] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:09.117275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00d7c0-d630-45fe-ba38-ffae18b662de None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.117557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00d7c0-d630-45fe-ba38-ffae18b662de None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.117826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00d7c0-d630-45fe-ba38-ffae18b662de None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.123413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 250/492] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:09.124310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00d7c0-d630-45fe-ba38-ffae18b662de None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:09.149414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:09.150042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.150375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.150642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.151282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:09.151575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:09.153046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59c2bd-ff95-4e53-bac0-976564f2a249 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:09.170215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 243/493] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:09.171760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.172911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.173341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.197137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:09.198514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:09.252021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:09.258494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00d7c0-d630-45fe-ba38-ffae18b662de None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:09.295625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 244/494] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:09.298294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:09.311581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:09.313042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.313932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.314538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.316110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=b981e25e0b8d46f298b05b261433de94) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:09.317030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:09.319564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:09.327980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 251/495] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:09 2022] GET /identity/v3/role_assignments?user.id=b981e25e0b8d46f298b05b261433de94&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:09.384730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:09.458580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b35206-29ef-4d0a-9196-eb6372f46f87 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:09.504851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:09.505531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.505805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.506071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.515330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:09.516955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:09.535743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:09.536737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:09.537710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 245/496] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:09.543100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:09.591253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:09.728426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a7c7-1ea7-4b32-93af-34d66d9a63a6 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:09.750589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70958b38-f0ad-40c0-93c8-895ee37152e2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:09.762599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:09.780731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:09.781375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.781921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.781921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.782701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:09.783011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:09.784551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:09.791591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:09.792462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:09.792907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:09.793359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:09.808476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:09.808971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 252/497] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 14:20:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:09.814852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:09.815509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:09.816581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:09.817497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:09.818178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 246/498] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:09.823734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:09.846574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:09.979323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99a326-ca34-4583-a4a4-6d5aa558f5f7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:10.029036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160cca-94ec-46bd-88a6-4ec681d350a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:10.029651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160cca-94ec-46bd-88a6-4ec681d350a6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.029924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160cca-94ec-46bd-88a6-4ec681d350a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.030186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160cca-94ec-46bd-88a6-4ec681d350a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.049775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e69b-5ba9-4a4d-a399-e7896ac3cba3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:10.093602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:10.094218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.094497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.094769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.095440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:10.095742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:10.101895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:10.122211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 247/499] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:10.128973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:10.170332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160cca-94ec-46bd-88a6-4ec681d350a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:10.172107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160cca-94ec-46bd-88a6-4ec681d350a6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:10.183987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:10.205537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:10.327655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301717ce-0e70-41e1-ad54-8a66a1d507ca None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:10.358956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 253/500] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 14:20:10.360334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458895e1-d7a0-4cba-84b5-290f3837adbd None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.365304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458895e1-d7a0-4cba-84b5-290f3837adbd None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.365641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458895e1-d7a0-4cba-84b5-290f3837adbd None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.367162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 254/501] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:10.367929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458895e1-d7a0-4cba-84b5-290f3837adbd None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:10.386591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516547d2-a855-4410-9884-a7265186572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:10.387358 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516547d2-a855-4410-9884-a7265186572e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.387640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516547d2-a855-4410-9884-a7265186572e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.388118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516547d2-a855-4410-9884-a7265186572e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.402338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516547d2-a855-4410-9884-a7265186572e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:10.403815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516547d2-a855-4410-9884-a7265186572e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-516547d2-a855-4410-9884-a7265186572e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:10.421736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:10.423176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:10.424570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:10.424570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:10.424570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:10.424570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 248/502] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:10.425738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.426017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.426286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.451191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:10.453199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:10.507108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458895e1-d7a0-4cba-84b5-290f3837adbd None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:10.538340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:10.551100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 249/503] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:10.552097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:10.561289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:10.562048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.562328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.562602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.568147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:10.585693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:10.587255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:10.605396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:10.607566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:10.609747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:10.609747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:10.609747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:10.609747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 255/504] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:10.618445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:10.759165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031ae40-1f4a-465e-b439-f48004827c4e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:10.768809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb3cba-1ce8-4141-b2b1-1290a4ed123a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:10.809844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:10.810758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.811049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.811331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.811987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:10.812292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:10.813929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:10.829925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 256/505] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:10.836143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:10.876212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:10.885732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:10.886695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:10.887139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:10.887610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:10.888443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:10.888934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:10.890600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:10.915510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 250/506] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:10.916666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:10.972993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb7b8-1527-4395-ba50-b1c7be824384 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.016721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:11.021852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.022692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.023066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.023517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.037513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.039192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.052291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382e02f-b97c-448f-afe0-d5c5f24cca1b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:11.054148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:11.055013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:11.055909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:11.060819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 257/507] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:11.062008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.101563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.102128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.102929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.103198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.104804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.106245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:11.122639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:11.125083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:11.127087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 251/508] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Nov 2 14:20:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:11.129139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.188674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:11.198322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdaf87-74c8-4462-a66c-0a9d2265879b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.246786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.247682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.248244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.248671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.259462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.261307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:11.277396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:11.279288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 258/509] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:11.281492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.301627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a9159-bdce-4c68-b02d-4b4ae086cfe3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.347287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.348200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.348480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.348766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.349508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:11.349821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.351506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.357262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:11.368176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 252/510] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:11.369138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.423732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833dcf3-4690-484f-a96c-6ac359682dfc None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.470177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.471946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.472400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.472978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.473683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:11.473983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.475363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6b54f-fec1-4f25-ac79-e7f6333f2906 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.481164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 259/511] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:11.481164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13f3d0-a875-4ca5-a665-287d209e47fc None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.481164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13f3d0-a875-4ca5-a665-287d209e47fc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.481164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13f3d0-a875-4ca5-a665-287d209e47fc None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.481164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 260/512] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:11.489434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13f3d0-a875-4ca5-a665-287d209e47fc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.500112 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6b430-d318-4e91-9875-7430ad854d76 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.509837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:11.547643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.548309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.548658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.548958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.550007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:11.550315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.551811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.576968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 253/513] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:11.576968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.619332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13f3d0-a875-4ca5-a665-287d209e47fc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.639204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:11.672440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.673079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.673360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.673630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.674448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=0435bede480b4f83aee82223ef80836f) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:11.674758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.676215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0eab6-520b-4f64-93e0-c9e612e3fafb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.682825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 261/514] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/role_assignments?user.id=0435bede480b4f83aee82223ef80836f&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:11.691783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.692276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.692754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.714884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489c0bd-419c-4544-ba72-7a539ebee004 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.726173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:11.728127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:11.766416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.767191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.767568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.768415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.769309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:11.769574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.771637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:11.779355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 254/515] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:11.780260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:11.853726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 262/516] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:11.927970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667f229-2317-4c8f-90fa-2fe1b80ec689 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:11.951993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:11.974988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:11.975664 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:11.975945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:11.976216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:11.990295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:11.992518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ba133c7e-c54a-4d28-ad96-31dade792a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:12.015948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:12.018480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:12.020594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:12.020594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:12.020594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:12.025035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 255/517] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:12.030185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.049934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:12.070142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fbbb9-40f0-4116-8814-9f6d43bdae2e None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.070529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fbbb9-40f0-4116-8814-9f6d43bdae2e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.070742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fbbb9-40f0-4116-8814-9f6d43bdae2e None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.072222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 256/518] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:12.078905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.078905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.078905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.099839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:12.101229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:12.167947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3acc14-508d-46cf-8865-95de77cf8562 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:12.188662 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 257/519] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:20:12.189801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.263121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:12.263884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.264253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.264605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.265383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:12.265774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:12.267237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.301048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 263/520] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:12.307597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.317650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:12.324200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27fc7f-94d0-499b-b42e-ee254b665b97 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:12.385693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:12.386659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.391269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.391705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.411453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:12.414047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.443857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:12.447421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:12.448401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:12.449498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:12.449498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:12.449498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:12.449498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 258/521] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:12.456651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.487799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7191-1d26-4241-ae2d-04ee5711cd3d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:12.538012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:12.538672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.538963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.539275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.556213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:12.558931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:12.583876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:12.585743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 264/522] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:12.589357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.652739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b03f5-4f3f-498a-be00-2cf9fcae3b76 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:12.698685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:12.699278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.699554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.700163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.701531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:12.701842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:12.703705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.718967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 259/523] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:12.720185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.739172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:12.748697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab140cf-2cb4-41d5-940e-44aa389ef144 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:12.785137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:12.791586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:12.792718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.793321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.793600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] PATH_INFO: `/v3/services/network` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.794230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:12.795784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:12.815612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:12.816980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:12.817947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:12.825724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 265/524] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:12.826836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.857988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691c474-6d25-46bc-8787-6ec313809331 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:12.918056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:12.919524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:12.920310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:12.920786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:12.921991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:12.922884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:12.925455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:12.942386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 260/525] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:12.943632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:12.990174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda7683-e353-4c68-a05f-a8982b2709ba None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:13.044626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:13.045745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:13.046268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:13.046831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:13.047708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:13.050349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:13.053324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:13.095366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e8346-df01-4d23-ac53-478447bed44d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:13.134361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:13.135343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:13.135811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:13.136294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:13.137176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:13.137671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:13.139293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:13.148409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 261/526] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:13.148409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:13.199291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 266/527] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:20:12 2022] POST /identity/v3/users => generated 254 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:13.200265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:13.272255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:13.305271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc4eee-baa5-42c1-98b5-a5c75bb5ffd8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:13.339324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e7109-1fca-49d2-95c0-dcf192fa499c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:13.369328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dce11b-8a4a-4e37-b64a-d82fb5b030ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:13.370018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dce11b-8a4a-4e37-b64a-d82fb5b030ea None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:13.370349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dce11b-8a4a-4e37-b64a-d82fb5b030ea None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:13.370632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dce11b-8a4a-4e37-b64a-d82fb5b030ea None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/ea17166a0ace485dbf6cadaae7731ed6/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:13.374328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dce11b-8a4a-4e37-b64a-d82fb5b030ea None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=ea17166a0ace485dbf6cadaae7731ed6, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:13.376033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dce11b-8a4a-4e37-b64a-d82fb5b030ea None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:13.386360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:13.405722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:13.406408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:13.406742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:13.407022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:13.407675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] RBAC: Adding query filter params (type=network) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:13.408032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:13.410129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389631e7-9c9c-41af-a7c8-f57ecffb9a97 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:13.418169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 267/528] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:13.496552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 262/529] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:13 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/ea17166a0ace485dbf6cadaae7731ed6/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:14.457876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b62445-b270-4737-8ebf-48c7a994ba12 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:14.458415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b62445-b270-4737-8ebf-48c7a994ba12 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:14.458686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b62445-b270-4737-8ebf-48c7a994ba12 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:14.460425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 268/530] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:14.466310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fea87-441d-4ba1-a834-b08455b351f2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:14.466855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fea87-441d-4ba1-a834-b08455b351f2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:14.466855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fea87-441d-4ba1-a834-b08455b351f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:14.488195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ed3fea87-441d-4ba1-a834-b08455b351f2 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:14.489502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ed3fea87-441d-4ba1-a834-b08455b351f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:14.600076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 263/531] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 14:20:14.836379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dcb2f-93e2-416d-821b-a53968b93d18 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:14.836379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dcb2f-93e2-416d-821b-a53968b93d18 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:14.836379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dcb2f-93e2-416d-821b-a53968b93d18 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:14.836379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 269/532] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:14.841447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:14.841878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:14.842124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:14.874280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:14.875593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:14.983274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 264/533] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:15.092636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dcb2f-93e2-416d-821b-a53968b93d18 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:15.237137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dcb2f-93e2-416d-821b-a53968b93d18 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:15.265220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:15.299503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:15.300214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:15.300672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:15.300979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:15.302076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:15.303668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:15.442848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 270/534] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:15 2022] POST /identity/v3/services => generated 251 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:15.470594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:15.545030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:15.608100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466b656-903e-433c-8154-b4ce89bfd677 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:15.660783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:15.661546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:15.661862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:15.662190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:15.676271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:15.681618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:15.682636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:15.683473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 265/535] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:15.689106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:15.828231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133c57e-4aee-494f-a4f3-465f50a82c5b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:15.875536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:15.876199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:15.876512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:15.876785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:15.878319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:15.878730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:15.880160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:15.897145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 271/536] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:15.901084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:16.035830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b95db-21c9-47e5-b763-3a23e0befc6f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:16.062539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:16.080990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:16.081964 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.082423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.082872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.094829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:16.098038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:16.118215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:16.119221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:16.120115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:16.120115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:16.120115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:16.120115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 266/537] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:16.126147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:16.169403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:16.250716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66728e4-4e95-42ec-bc85-6464913e86c6 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:16.301954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:16.302587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.302936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.303207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.303960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:16.304264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:16.305720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea722-3b70-4477-9d0e-7a19c49b9d58 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:16.322595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 272/538] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:16.328161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:16.388926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dcf5e-b396-4139-bcc5-10ed17e238c2 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.389304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dcf5e-b396-4139-bcc5-10ed17e238c2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.389495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dcf5e-b396-4139-bcc5-10ed17e238c2 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.390915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 273/539] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:16.397672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7ca9-f012-4481-b27b-5df4a04425ec None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.398040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7ca9-f012-4481-b27b-5df4a04425ec None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.398235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7ca9-f012-4481-b27b-5df4a04425ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.432783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2c1b7ca9-f012-4481-b27b-5df4a04425ec None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:16.434908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2c1b7ca9-f012-4481-b27b-5df4a04425ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:16.483675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde91c-5271-49f9-ae9c-4f63282a72c1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:16.530436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:16.531110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.531388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.531679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.537659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 274/540] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:16.539248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf81c09-e5ae-4f34-b639-cec3433835ff None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.539527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf81c09-e5ae-4f34-b639-cec3433835ff None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.540195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf81c09-e5ae-4f34-b639-cec3433835ff None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.548371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 275/541] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:16.548371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7624a-bf31-46cc-8ce4-c61135d305ad None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.548371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7624a-bf31-46cc-8ce4-c61135d305ad None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.548371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7624a-bf31-46cc-8ce4-c61135d305ad None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.548371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 276/542] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:16.552849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:16.554459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:16.556310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.557050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.557499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1b9e4c6d-361f-4647-bcc9-d63102e35c4b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:16.570814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:16.571821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:16.572747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 267/543] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:16.574314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.574701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.575073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.582990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:16.585452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:16.597334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:16.598794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:16.612078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:16.622516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:16.684418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 277/544] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:16.684418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:16.715284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 268/545] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 14:20:16.853227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755672b8-fb3c-498c-b083-9c61b27e5283 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:16.925865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:16.926483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:16.926792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:16.927150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:16.928172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:16.928477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:16.930051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:16.956347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 278/546] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 14:20:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:16.962529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.043255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.061900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:17.113281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd70c2-0980-4d28-bd39-acd425992da0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.149818 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.150441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.150715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.151162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.156959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2720-26bd-47e3-9b29-1db36dd2c300 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.162188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:17.163646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:17.178970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:17.184739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:17.185707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:17.185707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:17.185707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:17.185707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 269/547] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:17.185707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.194371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:17.203479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.209280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.209280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.209280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.209280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:17.209280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:17.288378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dbe78-954c-457d-9655-6fec72ff792d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.291124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 279/548] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:17 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:17.291965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.339324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.340524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.340984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.341474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.359759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:17.368813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:17.370005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:17.370863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:17.374392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 270/549] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:17.375485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.401275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:17.440209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a8149-e2c7-4099-af5e-932c327f56f3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.481445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:17.488616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.489771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.490045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.490312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.502735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:17.504111 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33d7fa-82d8-4c20-9b82-4e9db3feda65 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:17.508015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:17.508970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:17.509871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:17.510514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 280/550] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:17.511305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.550899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.551909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.552400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.552854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.554423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:17.554763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:17.556124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:17.572625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 271/551] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:17.573520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.627487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb33f5f-cc62-48bf-8375-653a882ad06d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.670529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.671139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.671436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.672063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.672831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:17.673483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:17.675247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:17.686468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:17.687407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 281/552] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:17.688160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.697771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919590e7-2387-4e80-b0d1-8c83d46b9fc9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.730719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:17.746794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.748000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.748486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.749350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.750497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:17.750999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:17.753334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:17.764923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 272/553] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:17.766142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.820141 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1d3b-2b8e-487e-bfc7-7cf75109a1fc None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:17.866497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:17.867168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:17.867404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:17.867663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:17.874870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:17.876306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:17.888628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:17.889640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:17.890665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 282/554] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:17.891152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:17.996788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921f090-26d4-49cc-87f3-a230b4867e31 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.032461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:18.034453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.035183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.035553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.035914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.047238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.049338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:18.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:18.066383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:18.067494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:18.067494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:18.067494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:18.068810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 273/555] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:18.069844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.091191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca662f-90f5-4581-8e3f-f24671212aa5 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.108950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:18.136662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.137949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.138501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.139217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.152990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.154699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:18.171509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:18.172706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:18.173853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:18.173853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:18.173853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:18.180404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 283/556] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:18.180404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.190994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852f96-c7f2-4796-a4ad-8e06fe495746 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.229198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.230060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.230375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.230740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.231576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:18.232013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.233579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.246603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 274/557] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:18.247657 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.310111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c396a12-31ae-4d33-86d0-c69fea432390 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.322691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:18.351782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.353205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.353972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.354573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.355470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:18.355939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.357226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:18.358006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.363986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac5c06-97dd-4ceb-acda-c2973336a843 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.374649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 284/558] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:18.375737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.404117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.405232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.405614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.406021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.407053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:18.407209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.409063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.422872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 275/559] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:18.423306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.506747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dfe19-c476-4e44-8182-9cdf32b63057 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.541174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446a30e-418b-46a4-ab8c-007eb7167371 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.548701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263530d-5e83-486f-ab8f-59f31a9a3e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.549638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263530d-5e83-486f-ab8f-59f31a9a3e98 None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.550187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263530d-5e83-486f-ab8f-59f31a9a3e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.550458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263530d-5e83-486f-ab8f-59f31a9a3e98 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/0435bede480b4f83aee82223ef80836f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.552491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263530d-5e83-486f-ab8f-59f31a9a3e98 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=0435bede480b4f83aee82223ef80836f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.553938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263530d-5e83-486f-ab8f-59f31a9a3e98 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.582605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.583210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.583506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.583796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.594044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.596093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:18.608274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:18.609544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 276/560] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:18.610490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.633687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:18.645043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 285/561] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:18 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/0435bede480b4f83aee82223ef80836f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:18.645043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360decc-7893-4267-b7be-fd382f893d4f None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.645043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360decc-7893-4267-b7be-fd382f893d4f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.645043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360decc-7893-4267-b7be-fd382f893d4f None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.645043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 286/562] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:18.645839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360decc-7893-4267-b7be-fd382f893d4f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.693395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:18.740881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd47479-0783-46f2-b61d-4c19aa63e986 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.780515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.781366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.781712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.782095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.791419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.797831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.799397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360decc-7893-4267-b7be-fd382f893d4f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5812cc1d-5946-4bc5-8c2b-770b16f4095a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:18.806979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:18.808097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:18.809157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 277/563] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:18.809790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.809868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.813030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.837881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:18.837881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:18.843966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:18.844628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:18.844909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:18.845228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:18.846751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:18.846751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:18.848260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:18.875601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 287/564] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:18.876778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.940971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 278/565] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:18.940971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:18.956994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.042105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.071200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7574a-9ec6-4d92-a505-7e86437a4ddd None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.106505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e7079-3820-44cc-9bf0-7a4804867b22 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.139056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.139676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.139953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.140221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.140951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:19.141288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.142828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.174086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.174710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.174981 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.175246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.179979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 288/566] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:19.185120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.298558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.301018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.309474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.337292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643714bb-272c-4cfc-ab6c-ccddc78da1ae None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.382521 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.383190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.383462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.383729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.397745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.399942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.403001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:19.418958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:19.421569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:19.422649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:19.422649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:19.422649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:19.422649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 289/567] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:19.422649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.439169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 279/568] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:20:19.439169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.545646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aae129-5822-426d-b402-08d49a9c37dc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.554954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.559575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5c12f-ecbc-435c-a0ab-7bc5a8ad82d1 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.589618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.590429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.590551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.590743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.591584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:19.591885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.593299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.605912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 280/569] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:19.606790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.609019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.609918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.610354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.610770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.612142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.613841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:19.623747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:19.625053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:19.626217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:19.629999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 290/570] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:19.631282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.702218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.735328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4adc2-dbd7-4350-90bd-c3e5cbb86423 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.753503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd68268-1ad4-4344-afe9-1cd3622d2f84 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.772624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.773286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.773564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.773830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.785455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.786973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.790212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.790838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.791144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.791415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.798289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.800506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:19.802282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:19.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:19.804315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:19.804315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:19.804315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:19.804315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 281/571] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:19.804315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-f5a6c338-a780-4fb6-9287-84c958aa2805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:19.818192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:19.819433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 291/572] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:19.820723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e7a0-6576-4f3e-b425-a673629d166a None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.821408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e7a0-6576-4f3e-b425-a673629d166a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.821408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e7a0-6576-4f3e-b425-a673629d166a None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.828050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 292/573] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:19.828050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e7a0-6576-4f3e-b425-a673629d166a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.832226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.925218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2564-5435-4220-a72e-532da8f95ad8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.949988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e7a0-6576-4f3e-b425-a673629d166a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:19.962945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.964006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.964236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.964561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.965337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:19.965703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.967187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:19.976316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 282/574] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:19.976316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:19.977462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:19.990758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:19.990758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:19.990758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:19.991115 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:19.991468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:19.991821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:19.993354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85850f-afe9-4177-b155-3f59a8cc5594 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.007947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 293/575] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:20.009602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572de0-7858-472d-983d-673c18e93663 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.010427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572de0-7858-472d-983d-673c18e93663 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.011187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572de0-7858-472d-983d-673c18e93663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.039721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-25572de0-7858-472d-983d-673c18e93663 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:20.041738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-25572de0-7858-472d-983d-673c18e93663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:20.085341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.108295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e42b-5a5f-4008-8c6e-b12ccc0e1c82 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.125947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 294/576] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:20.127660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572de0-7858-472d-983d-673c18e93663 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.157821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.158810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.159188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.159590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.160736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:20.161226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.162766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.185559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 283/577] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:20.186643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.262965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572de0-7858-472d-983d-673c18e93663 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.279603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.310029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa71c89-68c9-44df-a332-da8d59358429 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.312403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.314210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.315163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.315574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.316404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:20.316845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.318700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.331442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 295/578] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:20.331442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.355633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.356530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.356893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.357282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.365537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.367217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:20.380121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:20.381361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:20.382722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:20.388658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 284/579] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:20.389683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.392885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.461295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee33ae-bd65-4e8e-8cb7-49a4dad76028 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.496436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.497291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.497664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.498036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/c04dc4fcbdd6447daf928b3f14481011/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.500941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddf096-fcd2-4bdc-a3df-4c8401a6f7b2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.513256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=c04dc4fcbdd6447daf928b3f14481011, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.514726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.537708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.538319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.538591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.538860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.539550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] RBAC: Adding query filter params (interface=public, service_id=17428e006dca42f3a1846012935ecf78, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:20.539854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.541300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.549386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 285/580] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/endpoints?service_id=17428e006dca42f3a1846012935ecf78&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:20.550123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.605040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 296/581] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:20 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/c04dc4fcbdd6447daf928b3f14481011/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:20.605040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.648826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.655611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.705358 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2819c15-abac-4d99-acef-a64c3b3412fa None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.750600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.751515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.752131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.752560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.754639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:20.755190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.756991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.757444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67443395-2bbb-445e-996c-e5dac9a376ce None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.771950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 286/582] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:20.773286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.804146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.805190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.805618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.806076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.806935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.809523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:20.821907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:20.826503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:20.827633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 297/583] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:20.828615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:20.927423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8448d90-e83b-4715-bc12-ef3f3cf0e2cd None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.962678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.969938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:20.978748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0a65-d105-4a96-b260-25f5d952c9b9 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:20.982845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:20.983760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:20.984373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:20.984871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:20.985708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:20.988244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:21.003435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 287/584] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:21.003435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:21.043258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:21.044485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.044766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.045063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.046128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=ea17166a0ace485dbf6cadaae7731ed6) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:21.046432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:21.048210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a97a-a51c-4791-bf96-7b18a9bbe1e7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:21.058160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 298/585] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/role_assignments?user.id=ea17166a0ace485dbf6cadaae7731ed6&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:21.059698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242da31-1268-4657-a661-03d704a89c46 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.060060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242da31-1268-4657-a661-03d704a89c46 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.060540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242da31-1268-4657-a661-03d704a89c46 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.069178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 299/586] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:21.071130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.071928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.072229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.097911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:21.099508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:21.145803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822dddb-f6cf-405d-ae81-ac276f2f853a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:21.188392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:21.189560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.190079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.190554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.191470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] RBAC: Adding query filter params (name=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:21.191996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:21.193792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:21.199807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 288/587] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:21.206783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:21.207641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 300/588] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:21.277722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:21.332499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc6af-778f-48d6-9989-e8831c06bcfe None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:21.371430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:21.372044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.372317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.372579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.373228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] RBAC: Adding query filter params (type=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:21.373526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:21.374933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:21.381143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 289/589] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:21.386895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:21.481788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:21.501984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:21.502482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c0d73-c7db-4a87-894f-a0bc86c7cd03 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:21.539281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:21.539899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.540175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.540439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.541179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] RBAC: Adding query filter params (interface=public, service_id=2291d5dc761c4037a19b50b4fcf32ddd, region_id=RegionOne) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:21.541482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:21.543009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:21.553073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 301/590] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:21 2022] GET /identity/v3/endpoints?service_id=2291d5dc761c4037a19b50b4fcf32ddd&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:21.558568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:21.669915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeb2d4-8bef-468e-aed6-9361ec4c31bc None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:21.705438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:21.706317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.706607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.706875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.707485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:21.709596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:21.725037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 302/591] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:21 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:21.769721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc48cda-c935-4168-9766-0c8578f52152 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:21.798196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:21.827638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:21.828540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:21.828961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:21.829424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:21.848164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:21.853758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:21.854923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:21.855960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:21.858402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 290/592] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:21.864814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:21.886519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:21.994988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540c6d1-5b91-45ac-b81a-b112b93c8ccd None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:22.033671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:22.034286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:22.034567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:22.034837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:22.035501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:22.035802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:22.037294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:22.047043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 303/593] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:22.052770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:22.170794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c053cb9-d23a-4175-87f0-ecc243fa05ba None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:22.215767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:22.216412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:22.216688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:22.216956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:22.232867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:22.234387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:22.253346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:22.254324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:22.255233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:22.255233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:22.255233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:22.255818 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 291/594] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:22.260372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:22.356116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:22.393571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eff6f-fa00-4537-b792-2f37b31c494e None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:22.446303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:22.446941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:22.447220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:22.447492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:22.448204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:22.448719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:22.450126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:22.468452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 304/595] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:22.473837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:22.524431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:22.624867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926ebb-776e-4ad2-b924-0d881fee125b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:22.667163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:22.667821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:22.668101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:22.668373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:22.676945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:22.678444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:22.692953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:22.693995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:22.694938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 292/596] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:22.700263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:22.867398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692f8fe-cfd6-47df-860a-898b661e63bb None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:22.908433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:22.909072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:22.909535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:22.909989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:22.910444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:22.911475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:22.911956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:22.913694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:22.933022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 305/597] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 14:20:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:22.938691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:23.055823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:23.066208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b218-405f-4839-9842-8c5bde35e803 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:23.114761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749210c7-3d84-4b7e-b3c8-2739bd3f11cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:23.115803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749210c7-3d84-4b7e-b3c8-2739bd3f11cf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:23.116306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749210c7-3d84-4b7e-b3c8-2739bd3f11cf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:23.116780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749210c7-3d84-4b7e-b3c8-2739bd3f11cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:23.225339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749210c7-3d84-4b7e-b3c8-2739bd3f11cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:23.227575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749210c7-3d84-4b7e-b3c8-2739bd3f11cf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:23.338101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:23.352393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 293/598] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:20:23.358021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:23.389773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b167e-678a-48de-930d-2426a34c2f0b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:23.389968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b167e-678a-48de-930d-2426a34c2f0b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:23.390216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b167e-678a-48de-930d-2426a34c2f0b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:23.393187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 294/599] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:23.414774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:23.415084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:23.415326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:23.443960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:23.445177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:23.488732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8db2cd-357e-4a34-9b66-691a7d95d87f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:23.532097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:23.532427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:23.532689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:23.533427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:23.555899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:23.557590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:23.587224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:23.588262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:23.589197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:23.589197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:23.589197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:23.589197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 306/600] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:23.594245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:23.619103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 295/601] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:23.651870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:23.734458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e08a9-54ca-4dc5-bdcd-a3b2b3454d13 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:23.781694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:23.782215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:23.782495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:23.782880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:23.783685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:23.784103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:23.785507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:23.799096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 307/602] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:23.804849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:23.942033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:23.945731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9b888-ebcc-49a1-8fa8-0bf5b394001b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:23.989066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:24.015488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.016399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.016670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.016934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.031118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.032704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-282c719e-a0f4-48e9-91fb-92801d4a81db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:24.050228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:24.051194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:24.052087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 296/603] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:24.053050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e3c8-8169-4d12-a7c9-0d804a0aa764 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.053150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e3c8-8169-4d12-a7c9-0d804a0aa764 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.053393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e3c8-8169-4d12-a7c9-0d804a0aa764 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.058809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 297/604] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:24.058809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e3c8-8169-4d12-a7c9-0d804a0aa764 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.114202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675361ff-9538-49ea-bb1a-0d50c8262cb0 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.115902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:24.157105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a216693-d2aa-4080-a779-6a24b05424b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.158893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a216693-d2aa-4080-a779-6a24b05424b8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.158893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a216693-d2aa-4080-a779-6a24b05424b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.158893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a216693-d2aa-4080-a779-6a24b05424b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.169895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a216693-d2aa-4080-a779-6a24b05424b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-8a216693-d2aa-4080-a779-6a24b05424b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:24.174793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:24.175769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:24.176614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 308/605] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:20:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:24.177383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.177383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.177925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.184136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e3c8-8169-4d12-a7c9-0d804a0aa764 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.196705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:24.197928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:24.225442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.226097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.226371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.226637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.227604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:24.227927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.229380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449d2ed-c813-44c2-87d8-a02a60dbe77e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.241536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 298/606] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:24.242770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb1f6-3ff6-4170-8bea-29f2f47a622d None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.243048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb1f6-3ff6-4170-8bea-29f2f47a622d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.243318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb1f6-3ff6-4170-8bea-29f2f47a622d None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.244655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 299/607] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:24.245200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eb1f6-3ff6-4170-8bea-29f2f47a622d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.287915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 309/608] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:24.289087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.301096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:24.384127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eb1f6-3ff6-4170-8bea-29f2f47a622d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.408889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:24.419623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75102a16-1354-4701-baf7-7af5e2ec223b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.430565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.431857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.432630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.433142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.434159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:24.434760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.437074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f440a06-61cc-4ee5-a562-05ab7880f187 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.448918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 300/609] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:24.450532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.451006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.451464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.464743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.465485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.465834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.466190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.466989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=0435bede480b4f83aee82223ef80836f) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:24.467394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.469655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.477505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:24.479140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:24.484016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 310/610] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/role_assignments?user.id=0435bede480b4f83aee82223ef80836f&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:24.485091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.590355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 301/611] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:24.592268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.674458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec33ad-864f-4e69-b205-03ec35795cb7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.696294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:24.729475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.730234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.730583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.730926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.747669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.749261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:24.773568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:24.774570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:24.775468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:24.775468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:24.775468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:24.775468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 311/612] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:24.782288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.788841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f90a9-7427-4a80-887c-ba04a26ae60b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.794075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:24.835266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.835905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.836222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.836655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.837349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.839070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1d92c275-e673-4424-ba9b-89902ca20084 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:24.852064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:24.853376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 14:20:24.854200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:24.857079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 302/613] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:24.858650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d57a-c554-4dbc-843a-82375bfcc5ce None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.859286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d57a-c554-4dbc-843a-82375bfcc5ce None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.859559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d57a-c554-4dbc-843a-82375bfcc5ce None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.861691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 303/614] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:24.862393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893d57a-c554-4dbc-843a-82375bfcc5ce None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:24.916463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4c9cf-8cb7-4101-8101-28dcd56dfa16 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:24.964948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:24.965586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.965861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.966127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:24.966766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:24.967062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:24.968416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9042d-62ed-4255-b8aa-b59f0ce8945b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:24.988027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 312/615] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:24.988027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:24.988027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:24.988027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.005331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:25.009301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:25.010574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:25.013038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893d57a-c554-4dbc-843a-82375bfcc5ce None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.052228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.052842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.053264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.053540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.054179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:25.054484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.055880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.061442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 304/616] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 14:20:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:25.062396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.101528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:25.116592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 313/617] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:25.116592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.194693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0ac1b-3e97-49f6-a7b1-986ad8ae76b9 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.244723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.245908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.246711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.247480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.248328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.249244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05724-6264-49e5-b158-2c73aacf761b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.250193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:25.265747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:25.266797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:25.268101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:25.271190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 305/618] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:25.272384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.295893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.296710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.297117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.297491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.307750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.309441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.316727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:25.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:25.333230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 314/619] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:25.334208 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.413384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:25.413997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3e858-3e7f-4cd4-934f-66533355c0e7 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.464212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.465302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.465798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.466293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.467511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:25.468068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.468170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c672d1-53e9-49ea-ae1f-bdfd0a5b0fd0 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.471293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.485257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 306/620] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:25.485257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.514064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.514666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.517287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.517287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.517287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:25.517287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.517821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.527692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 315/621] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:25.527692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.623279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:25.637047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31308-f79d-4b5b-ba61-1791a267c973 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.663988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81bc9-850b-4de3-982e-c8a4fa4f1509 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.679760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.680466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.680748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.681171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.682029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:25.682411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.684057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.693415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:25.705785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.706582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.706855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.707120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] PATH_INFO: `/v3/services/network` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.707710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.709197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.712274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 307/622] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:25.713307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:25.722613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:25.723980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:25.724955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 316/623] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:25.725839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.833797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed194f5-8378-4dcd-ad17-2232798710df None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.860234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40397-9715-413e-bc35-36bead16c97d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:25.874727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.875348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.875621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.875888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.877095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.878483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.909553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:25.910204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:25.910487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:25.910765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:25.911475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:25.911779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:25.913220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:25.918323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 317/624] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:25.919126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.989232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 308/625] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:25.992245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:25.998973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:26.001663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:26.122828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fca09-ce74-4598-94c5-c0b800edba37 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:26.184457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:26.184992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.185293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.185592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.186812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39606e-daeb-480d-9299-fd457d200766 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:26.241667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:26.242289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.242705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.242816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.243463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] RBAC: Adding query filter params (name=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:26.243763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:26.246670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:26.256551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 309/626] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:26.260646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:26.287118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:26.289100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:26.361647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:26.391891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:26.409833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d51040-8e28-4833-984b-27d506aeb5dc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:26.440850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 318/627] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:20:26.446674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:26.450648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:26.450960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.451338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.451708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.452463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] RBAC: Adding query filter params (type=network) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:26.452867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:26.454718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:26.467485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 310/628] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:26.468627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:26.594154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5298-e3ff-4e81-9f68-017955c081dc None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:26.630311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008791-3d27-427d-a90b-5f797b70ed29 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:26.661131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:26.662045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.662467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.662881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.683490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:26.686110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:26.690214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:26.711151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:26.711340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:26.711340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.711340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.711340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.711688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:26.711688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:26.715105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:26.716257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:26.717447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:26.717447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:26.717447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:26.721189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 319/629] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:26.722037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:26.845211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0c5f1-f7cf-4226-b8aa-0c81b139ded7 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:26.846633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 311/630] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:26 2022] POST /identity/v3/endpoints => generated 337 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:26.910028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:26.910674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.911010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.911273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.911932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:26.912222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:26.913701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a48604-1075-459e-a92f-7ffb262cc005 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:26.935577 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 320/631] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:26.941421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:26.951141 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89423-c130-4742-b581-bfac3c372335 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.951543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89423-c130-4742-b581-bfac3c372335 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.951697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89423-c130-4742-b581-bfac3c372335 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:26.953535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 321/632] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:26.963240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:26.963418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:26.963696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.003086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:27.004323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:27.105444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:27.125994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 322/633] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:27.152053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6c14d-9cc6-4035-a66f-25bb621e813e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:27.193271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:27.194593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:27.195018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:27.195438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.202897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:27.205147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:27.217829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:27.218925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:27.220200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:27.222476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 312/634] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:27.228579 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:27.348853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e8aec-811b-425a-b9e4-64f82bcb158d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:27.388232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:27.389207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:27.389662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:27.390107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.390973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:27.391469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:27.393667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:27.406510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 323/635] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:27.412947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:27.462955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:27.476929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:27.521623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:27.559495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3b1fb-bd57-4683-b5a1-015638096c16 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:27.583573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d8bde-2ef6-4958-bc7f-7c6f42c0189b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:27.607546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:27.608789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:27.609696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:27.610153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.612022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=c04dc4fcbdd6447daf928b3f14481011) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:27.612518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:27.615120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:27.622260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:27.622975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:27.623322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:27.623663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.629220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:27.629220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:27.629593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 313/636] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:27 2022] GET /identity/v3/role_assignments?user.id=c04dc4fcbdd6447daf928b3f14481011&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:27.639683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:27.641023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:27.643960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:27.646638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 324/637] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:27.653048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:27.776939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621567c-32c6-46e2-a4a1-2764c6ce774c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:27.790130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:27.821777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:27.822401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:27.822675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:27.822946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.823575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:27.823940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:27.825359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cf923-4c6f-4f92-867a-d215f5355c3f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:27.831030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 314/638] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:27.836710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:27.960388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb441-8be7-4202-bbbd-74c5b34f253d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:27.969885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:27.996026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:27.996637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:27.997041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:27.997317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:27.997954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:27.998254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:27.999609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba2bb2-e9d8-45a2-9b5a-98215c78424c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:28.005583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 325/639] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:28.508857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c28a8-96ed-482a-9806-62c4b70da51f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:28.509463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c28a8-96ed-482a-9806-62c4b70da51f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:28.509463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c28a8-96ed-482a-9806-62c4b70da51f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:28.511118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 315/640] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:28.517666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48185f-edf0-4a13-9184-80c228cab060 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:28.518168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48185f-edf0-4a13-9184-80c228cab060 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:28.518168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48185f-edf0-4a13-9184-80c228cab060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:28.537296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3c48185f-edf0-4a13-9184-80c228cab060 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:28.538498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3c48185f-edf0-4a13-9184-80c228cab060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:28.618501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 326/641] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:28.706706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263d430-91d9-44a7-b4eb-b42ea5db9dff None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:28.707129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263d430-91d9-44a7-b4eb-b42ea5db9dff None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:28.707502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263d430-91d9-44a7-b4eb-b42ea5db9dff None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:28.709242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 316/642] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:28.714440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:28.714723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:28.714996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:28.737054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:28.738252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:28.811774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 327/643] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:28.909054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263d430-91d9-44a7-b4eb-b42ea5db9dff None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:29.036217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263d430-91d9-44a7-b4eb-b42ea5db9dff None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:29.074784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:29.075307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:29.075582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:29.075862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:29.076576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:29.078304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:29.120716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:29.157059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 317/644] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:28 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:29.167415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:29.258713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e89d-2ae4-4542-9c94-1d2c7331c88a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:29.305648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:29.306562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:29.306985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:29.307394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:29.308324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:29.309926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:29.320819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:29.322161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:29.323229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:29.325945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 328/645] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:29.334261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:29.379125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:29.456782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d69398-6e15-4166-88bd-55c2e0e81cf1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:29.502429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:29.503060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:29.503343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:29.503616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:29.504258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:29.504560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:29.505973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98202f96-177b-4a4d-b4cb-d8e7c5b9d8da None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:29.515798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 318/646] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:29.518279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:29.652388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:29.662768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0a67c-a460-4ed8-a3fc-4c9b5ba3fbc5 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:29.702313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:29.702934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:29.703216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:29.703491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:29.704154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:29.704848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:29.706826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40fb12-b0a1-4ec8-bbd3-f8ccbbe9b526 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:29.712239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 329/647] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:30.090775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae9b8e-5518-4201-babb-dbd33ac367e2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.091159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae9b8e-5518-4201-babb-dbd33ac367e2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.091380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae9b8e-5518-4201-babb-dbd33ac367e2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.092978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 319/648] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:30.099497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0c463-f205-41db-9600-69bee40badc9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.099861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0c463-f205-41db-9600-69bee40badc9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.100105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0c463-f205-41db-9600-69bee40badc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.120648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-08b0c463-f205-41db-9600-69bee40badc9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:30.121886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-08b0c463-f205-41db-9600-69bee40badc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:30.198304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 330/649] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:30 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:30.353419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84166d-74bf-4414-b8ec-eaba448ac2aa None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.353750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84166d-74bf-4414-b8ec-eaba448ac2aa None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.353959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84166d-74bf-4414-b8ec-eaba448ac2aa None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.355564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 320/650] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:30.361625 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.362016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.362291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.382003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:30.383194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:30.460015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 331/651] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:30 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:30.492259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84166d-74bf-4414-b8ec-eaba448ac2aa None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:30.611772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:30.618308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84166d-74bf-4414-b8ec-eaba448ac2aa None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:30.659453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:30.660175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.660568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.660930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.673844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:30.679051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:30.680075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:30.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 321/652] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:20:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:30.691475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:30.721661 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:30.763182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:30.837780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f427-5708-46df-9a4e-4962ef6f117b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:30.888462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:30.888462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.888462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.888462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.888969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:30.889287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:30.891759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:30.905735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 332/653] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 14:20:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:30.912921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:30.944560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ff24d-b6c7-4d29-91ca-11f782d976de None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:30.994656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:30.996048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:30.996819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:30.997442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:30.998576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:31.000950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:31.021170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:31.046338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1a9ce-b2b7-4e41-a9b1-77f86008d12c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:31.097327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 322/654] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:30 2022] POST /identity/v3/services => generated 241 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:31.111401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:31.112733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.113269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.113773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.135652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:31.138488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:31.168916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:31.170006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:31.170934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:31.170934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:31.170934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:31.171690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 333/655] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:31.180894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:31.219525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:31.345788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67511ae-276d-4e44-a4da-df1ed60b5c42 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:31.393796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:31.394436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.394707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.394973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.395628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:31.395921 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:31.397433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:31.411366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 323/656] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:31.417347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:31.534427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c570d5-a539-4862-b2e4-a0ea322c12b5 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:31.566020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:31.575776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:31.576466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.576773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.577082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.588887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:31.590390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-ccc34643-c411-485b-855b-a681e7aecca1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:31.605630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:31.606809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:31.607979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:31.611546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 334/657] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:20:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:31.617895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:31.715910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:31.741933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88bb5e-388a-457b-94b8-1e1e342db7de None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:31.781680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:31.782826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.783110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.783452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.784390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:31.784691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:31.786380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:31.787179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107da4a6-91b1-4b4d-86b6-f4de069e38fe None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.787617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107da4a6-91b1-4b4d-86b6-f4de069e38fe None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.787734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107da4a6-91b1-4b4d-86b6-f4de069e38fe None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.789355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 335/658] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:31.795148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c064-249a-42da-8707-76606e44b607 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.795435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c064-249a-42da-8707-76606e44b607 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.795700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c064-249a-42da-8707-76606e44b607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.807519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 324/659] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 14:20:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:31.814284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:31.816081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-11c3c064-249a-42da-8707-76606e44b607 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:31.817398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-11c3c064-249a-42da-8707-76606e44b607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:31.913432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 336/660] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:31 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:31.930406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abdf87-7420-4120-9cff-1f83abce445b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:31.968029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:31.968670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:31.968943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:31.969244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:31.972203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:31.980881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:31.982381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-588e3b65-eef2-41ce-b938-4982adbeddb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:31.999750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:32.001063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:32.003091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:32.003091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:32.003091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:32.003091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 325/661] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:32.009573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c064-249a-42da-8707-76606e44b607 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.135903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c064-249a-42da-8707-76606e44b607 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.150552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1983-c2ae-4ab7-9b47-5b12bcf0ff44 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.151024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1983-c2ae-4ab7-9b47-5b12bcf0ff44 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.151135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1983-c2ae-4ab7-9b47-5b12bcf0ff44 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.152711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 326/662] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:32.159138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.159548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.160034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.166532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:32.177884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.178529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.178851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.187808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.187808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:32.187808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.188076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:32.188154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.188224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:32.195420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 337/663] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:32.201203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.268173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 327/664] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:32 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:32.269338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.325752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e727cc-a837-4ef7-9d9c-f869785bb98b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.367191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.368132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.368608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.369092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.377047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.378786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:32.391056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:32.392792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:32.393907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:32.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 338/665] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:32.406125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.421967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e81e9-c6e3-40f6-9715-377d308a79e7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.448770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:32.455111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:32.472959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.474268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.474726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.475175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.475942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.479460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:32.489936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:32.497876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:32.498793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 328/666] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 14:20:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:32.499792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.540560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f27611-df53-4dc3-880b-011a5b70492f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.586044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.586678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.586954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.587222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.587917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:32.588221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.590114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.604162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 339/667] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:32.605018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.648806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba152ce-26a7-42a8-b9da-2bf4b82b551f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.690968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.691504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.691778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.692045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.692678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:32.692996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.694398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.700324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 329/668] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:32.701414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.730675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:32.737805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13a4f-dfd5-4c9e-85cb-9a826adf740d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.761199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:32.782097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.783221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.783714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.784312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.785567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.786525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.823878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e29d0-7687-4b44-9347-d9a4c43fe1c0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:32.859535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:32.860979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:32.861432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:32.861899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/c04dc4fcbdd6447daf928b3f14481011/roles/a889c76e5a74438a9278ae93b3c0d84a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:32.864371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=c04dc4fcbdd6447daf928b3f14481011, role_id=a889c76e5a74438a9278ae93b3c0d84a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:32.864501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 340/669] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:32 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:32.865340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:32.866624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:32.977659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 330/670] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:32 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/c04dc4fcbdd6447daf928b3f14481011/roles/a889c76e5a74438a9278ae93b3c0d84a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:20:33.030434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53052e0f-f7c2-43bd-968f-0d9edec7038a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:33.076252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:33.076877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:33.077188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:33.077456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:33.078203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:33.078474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:33.080011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:33.086296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 341/671] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:33.093478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:33.225094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a2e1a-a243-4085-9377-8df7c8afb248 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:33.233671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:33.265452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:33.268377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:33.268825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:33.269274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:33.270206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] RBAC: Adding query filter params (interface=public, service_id=43ed535e2fa142d1afc7783c0f3cf907, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:33.270653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:33.272281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94802b59-950c-4948-a267-efbd25992a03 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:33.281927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 331/672] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:33 2022] GET /identity/v3/endpoints?service_id=43ed535e2fa142d1afc7783c0f3cf907&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:33.284661 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:33.306929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:33.397776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869bad5-0161-4c05-b646-ed60bd8f79a2 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:33.439514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500201-2a8d-45e1-b347-9d9f4d66daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:33.440623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500201-2a8d-45e1-b347-9d9f4d66daba None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:33.441149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500201-2a8d-45e1-b347-9d9f4d66daba None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:33.441624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500201-2a8d-45e1-b347-9d9f4d66daba None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:33.442466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a500201-2a8d-45e1-b347-9d9f4d66daba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:33.444036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a500201-2a8d-45e1-b347-9d9f4d66daba None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:33.451237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 342/673] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:33 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:20:33.623227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b891b-faca-4ee9-9c80-ac30640d79f4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:33.623569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b891b-faca-4ee9-9c80-ac30640d79f4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:33.623784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b891b-faca-4ee9-9c80-ac30640d79f4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:33.625482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 332/674] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:33.631609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:33.631976 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:33.632144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:33.652647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:33.653934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:33.729855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 343/675] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:34.004193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b891b-faca-4ee9-9c80-ac30640d79f4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:34.121405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b891b-faca-4ee9-9c80-ac30640d79f4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:34.158815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:34.159441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:34.159717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:34.159982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:34.160563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:34.161975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:34.171654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:34.173466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:34.174993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 333/676] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:34.182438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:34.201954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:34.300341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f2017-d244-41f0-87e6-081c7a9e1527 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:34.336304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:34.336918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:34.337273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:34.337546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:34.338218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:34.338519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:34.339881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:34.345544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 344/677] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:34.351513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:34.393952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:34.475424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751527-14e2-4cc4-90a3-49f85b3ff256 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:34.515810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:34.516594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:34.517477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:34.517758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:34.518423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:34.518803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:34.520251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:34.526470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 334/678] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:34.532463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:34.633376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:34.641261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605407-5b5c-4d26-a012-47086998680f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:34.678419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:34.679025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:34.679302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:34.679573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:34.680270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] RBAC: Adding query filter params (interface=public, service_id=85bfdf5a1425432198d6078fff6cf332, region_id=RegionOne) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:34.680574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:34.682138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f73b-bccd-48e8-bb20-248124888396 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:34.687803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 345/679] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:34 2022] GET /identity/v3/endpoints?service_id=85bfdf5a1425432198d6078fff6cf332&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:34.698119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:34.806862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:34.810874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de1441-d95c-49e0-aa5b-9531b8a646ec None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:34.848974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656884e1-09c6-4abc-9bd1-883c43166ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:34.849633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656884e1-09c6-4abc-9bd1-883c43166ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:34.849911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656884e1-09c6-4abc-9bd1-883c43166ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:34.850181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656884e1-09c6-4abc-9bd1-883c43166ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:34.850787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656884e1-09c6-4abc-9bd1-883c43166ef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:34.852426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656884e1-09c6-4abc-9bd1-883c43166ef5 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:34.859584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 335/680] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:34 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:20:35.292300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf702c0e-fc6c-4743-9195-3bf80ddb771d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.292859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf702c0e-fc6c-4743-9195-3bf80ddb771d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.292859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf702c0e-fc6c-4743-9195-3bf80ddb771d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.294535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 346/681] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:35.300743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e254f5-cd39-4c17-a816-c052a6e720b5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.301265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e254f5-cd39-4c17-a816-c052a6e720b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.301342 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e254f5-cd39-4c17-a816-c052a6e720b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.321388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-25e254f5-cd39-4c17-a816-c052a6e720b5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:35.322799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-25e254f5-cd39-4c17-a816-c052a6e720b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:35.403303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 336/682] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:35.499714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771067a8-03bf-45f9-9b61-b54fef639b8c None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.500040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771067a8-03bf-45f9-9b61-b54fef639b8c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.500290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771067a8-03bf-45f9-9b61-b54fef639b8c None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.502461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 347/683] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:35.507973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b55f29-a08f-4a98-8957-d821fea65cd9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.508264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b55f29-a08f-4a98-8957-d821fea65cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.508538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b55f29-a08f-4a98-8957-d821fea65cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.528695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f5b55f29-a08f-4a98-8957-d821fea65cd9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:35.529931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f5b55f29-a08f-4a98-8957-d821fea65cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:35.608611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 337/684] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:35.696359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771067a8-03bf-45f9-9b61-b54fef639b8c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:35.796701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0c0bb-f6d8-4295-8e4b-12ff4722b7fe None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.797041 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0c0bb-f6d8-4295-8e4b-12ff4722b7fe None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.797418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0c0bb-f6d8-4295-8e4b-12ff4722b7fe None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.804072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 338/685] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:35.806009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.806009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.806488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.814185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771067a8-03bf-45f9-9b61-b54fef639b8c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:35.826946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:35.828359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:35.849114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:35.865715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:35.866619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:35.867156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:35.867703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:35.868549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:35.871173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:35.883140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:35.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:35.885085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:35.890974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 348/686] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:35.897672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:35.909431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:35.913646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 339/687] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:35.923450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.037980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455bd00-7073-4a6c-97a8-0877b7ceb29b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.048776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545b4e-6c07-4b55-a626-381e0b15b20b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.086830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.087822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.088283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.088783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.096897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.097724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.098122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.098752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.099547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:36.100309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.101850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.104183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:36.113056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 349/688] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:36.113248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:36.114367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:36.115382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 340/689] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:20:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:36.118706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.123088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.148435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:36.214942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:36.265179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fbb3-afe6-4f01-bc4b-a3063f8ddaa5 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.267079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5dc2f-c60d-405f-bb83-de8251fb6e24 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.308369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.309071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.309275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.309542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.310191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:36.310467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.311830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.318130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 350/690] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:36.318850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.326495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.328271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.328778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.329355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.330728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:36.331615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.333518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.345653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 341/691] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:36.346910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.442092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:36.442424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba492a0-e8dd-47bc-9161-5707759d23f8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.469336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976b372-4654-4c89-bb4b-763c6cb9159b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.476449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:36.483955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.484582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.484859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.485841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.486519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.488423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:36.498464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:36.499465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:36.500415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 351/692] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:36.502597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.514502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.515114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.515398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.515647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.516936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] RBAC: Adding query filter params (interface=public, service_id=81f467cb0d2c47da9ed603cf6d2d515d, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:36.517390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.518817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.524835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 342/693] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/endpoints?service_id=81f467cb0d2c47da9ed603cf6d2d515d&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:36.525695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.616948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337a35d-9c93-47ae-97f5-f594ca6cabfb None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.631355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd665de4-f74c-481f-ab04-1b67a406610a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.656356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.657750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.657915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.658085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.667135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.667742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.668013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.668276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.668896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:36.669217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.670242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.670579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.671831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.676737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 343/694] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:36.678327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.683088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:36.689236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:36.690224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:36.691126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:36.691126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:36.691126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:36.691126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 352/695] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:36.691126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.746185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:36.798591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04acbc-7eb3-44e1-a447-0ac1c104baba None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.812967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336febaf-8edf-4870-b597-80f5cf94a483 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:36.837254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.837888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.838166 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.838440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.839057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.840469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.849134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 344/696] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:20:36.849134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.868393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:36.869383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:36.869866 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:36.870291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:36.871246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:36.871746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:36.873540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:36.881637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 353/697] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:36.892656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:36.953747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:37.013772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf57d1-561c-4f44-ba2e-75495ddf7f8f None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.026207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32886d5-b0a6-4c67-bbe8-aef41764f701 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.045378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:37.062635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:37.063281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.063607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.063840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.064697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:37.065175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:37.066593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:37.081681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 345/698] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:37.082596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8eae-5693-4759-8a79-73b6020bb6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:37.083215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8eae-5693-4759-8a79-73b6020bb6cd None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.083485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8eae-5693-4759-8a79-73b6020bb6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.083750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8eae-5693-4759-8a79-73b6020bb6cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.084440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a8eae-5693-4759-8a79-73b6020bb6cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:37.085833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a8eae-5693-4759-8a79-73b6020bb6cd None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:37.086683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:37.170098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 354/699] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:37.171486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d278c-6751-459b-b2e8-cc51bab009d8 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.171777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d278c-6751-459b-b2e8-cc51bab009d8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.172051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d278c-6751-459b-b2e8-cc51bab009d8 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.177046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 355/700] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:37.180138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.180516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.180833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.202441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:37.203852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:37.209925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acbedd-64d1-46bb-a98b-fb2a50b6917c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.242869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:37.259915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:37.261145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.262634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.264016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.274375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:37.277256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:37.292984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:37.294121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 14:20:37.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:37.298545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 346/701] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:20:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:37.304866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:37.317187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 356/702] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:37.431167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e3851-c5a1-46e1-bb6b-5be40c5ad84c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.469184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:37.470447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.470909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.471400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.472318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:37.472806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:37.474456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:37.494062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 347/703] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 14:20:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:37.501216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:37.609134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:37.618200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770dce7-24bf-4360-af60-8fd663e324d1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.636603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:37.668791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:37.669905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.670190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.670462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.696436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:37.770429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02a456-e000-4e0a-9647-e793a5708cfa None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.780304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:37.781772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:37.817536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:37.818817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:37.825231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:37.825231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:37.825231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:37.825231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:37.836581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:37.838106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:37.839005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:37.845470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 348/704] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:37.846976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:37.956053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:37.965149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 357/705] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:20:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:20:37.970108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:37.989819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f625907-3f87-496c-8569-74a3f865bda2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:37.993181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:38.046860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.048959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.049738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.050364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.051619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:38.052573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.054419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.062655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 349/706] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:38.071405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:38.101635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72132e-c8c9-4417-b9ba-c9c65bdb034d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:38.151555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.152227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.152508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.152782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.171139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.172973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:38.192865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:38.193996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:38.195111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:38.195111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:38.195111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:38.199314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 358/707] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:38.200341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:38.227657 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f7b56-64be-406c-afeb-9d76f9e05e67 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:38.269635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:38.289258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.289972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.290158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.290461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.291075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:38.291379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.292927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.299469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 350/708] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:38.308930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:38.338968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2950e7-5695-43e0-ada2-e57b8eac3110 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:38.353114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:38.379488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.380277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.380560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.380831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.381666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:38.381899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.383341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.397612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 359/709] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:38.403620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:38.456336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cabfa-37ea-41aa-84c4-3798a5473d3a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:38.505825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.507899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.508405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.508869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.509780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.511576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.551858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24701ac6-84d5-43ea-a1f1-e340580ee14c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:38.568410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:38.613673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.614358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.614638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.614915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.623203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.627590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.633867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 351/710] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:38.639735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:38.640737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:38.642864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 360/711] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:38.650441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:38.783248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:38.815642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a5dd9-a31e-47bb-90bb-a6fa897171db None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:38.856981 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:38.857645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:38.857916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:38.858251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:38.859137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:38.859390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:38.861669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1851a6-40fd-4247-acb2-1bd202a7d441 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:38.875615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 352/712] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:38.886074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:39.066064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cfd04-0ca7-4843-81b4-c9dcb8a4fa7b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:39.106368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:39.106971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:39.107250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:39.107531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:39.108228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] RBAC: Adding query filter params (role.id=a889c76e5a74438a9278ae93b3c0d84a, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=c04dc4fcbdd6447daf928b3f14481011) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:39.108528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:39.110696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520086ef-0016-49aa-b090-e1f0d1e79bb3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:39.118550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 361/713] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:20:38 2022] GET /identity/v3/role_assignments?user.id=c04dc4fcbdd6447daf928b3f14481011&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=a889c76e5a74438a9278ae93b3c0d84a => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:39.338966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daebd8-6fd4-45e7-829c-9dfb7b574788 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:39.339450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daebd8-6fd4-45e7-829c-9dfb7b574788 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:39.339586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daebd8-6fd4-45e7-829c-9dfb7b574788 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:39.341370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 353/714] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:39.347599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f6044-ff38-486c-8f2e-3621342c7e11 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:39.348453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f6044-ff38-486c-8f2e-3621342c7e11 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:39.348453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f6044-ff38-486c-8f2e-3621342c7e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:39.369054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-914f6044-ff38-486c-8f2e-3621342c7e11 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:39.370329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-914f6044-ff38-486c-8f2e-3621342c7e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:39.402218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:39.473855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 362/715] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:39.678421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9efa1-df1d-49cc-a49a-faac7afc79a4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:39.678421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9efa1-df1d-49cc-a49a-faac7afc79a4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:39.678421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9efa1-df1d-49cc-a49a-faac7afc79a4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:39.679447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 354/716] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:39.686151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:39.686888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:39.687368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:39.713515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:39.715226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:39.803210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 363/717] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:39.973939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9efa1-df1d-49cc-a49a-faac7afc79a4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.184872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9efa1-df1d-49cc-a49a-faac7afc79a4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.199512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.259328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.260309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.260795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.261303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.263226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.266560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:40.291925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:40.293896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 14:20:40.295514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:40.301370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 355/718] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:40.310889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.334855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:40.359302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:40.373554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbab5d-ad52-45f0-9bce-74627b102e79 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.425732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.427109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.428264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.428798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.429700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.431958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:40.444437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:40.445643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:40.446508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:40.450525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 364/719] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:40.459829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.480013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa57fc2-55ea-4a23-9eb1-2fcf4e1e4919 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.536481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.537663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.538018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.538366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.539387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:40.539767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.541570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.552570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 356/720] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:40.554648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.627730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eef5aa-8665-40d4-b376-41e6f8962b4d None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.659608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:40.678110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.679078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.679751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.679874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.680628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:40.680935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.682369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.688788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 365/721] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:40.698213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.709935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636da700-3f5e-4f19-92d6-86126f3f485d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.741527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:40.762793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.764160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.764610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.766643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.766643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:40.766643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.768398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.775175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 357/722] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 14:20:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:40.780587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:40.878096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134abe88-b8b1-4e4c-b1ff-4ab66b11ca62 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.921853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9001a4-ec27-4b9f-94ce-b6ae83b44bc1 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:40.956162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.957450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.958099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.959041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.961336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:40.962087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.964864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2125ba1-5025-4f4a-9e43-793a4a304454 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.973783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a0ca-81d9-4b0f-b12f-793f35c89961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:40.979595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 366/723] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:40.979811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a0ca-81d9-4b0f-b12f-793f35c89961 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:40.979811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a0ca-81d9-4b0f-b12f-793f35c89961 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:40.979811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a0ca-81d9-4b0f-b12f-793f35c89961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:40.979811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1a0ca-81d9-4b0f-b12f-793f35c89961 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:40.982628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1a0ca-81d9-4b0f-b12f-793f35c89961 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:40.992790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:41.122304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 358/724] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:41.704424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a842cb-21df-4854-b2f7-d8e8f84e6cd2 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:41.704985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a842cb-21df-4854-b2f7-d8e8f84e6cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:41.704985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a842cb-21df-4854-b2f7-d8e8f84e6cd2 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:41.706691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 367/725] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:41.712239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:41.713142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:41.713637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:41.733583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:41.736108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:41.767200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b6bf-3ecb-494e-be31-2f271d72cce7 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:41.767515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b6bf-3ecb-494e-be31-2f271d72cce7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:41.767754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b6bf-3ecb-494e-be31-2f271d72cce7 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:41.769364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 368/726] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:41.774814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:41.775099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:41.775384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:41.796231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:41.798141 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:41.828842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 359/727] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:41.880209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 369/728] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:42.107559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.168892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.185416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:42.254599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1652e5-2e8b-418e-997d-fce43e83e7e3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:42.281836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:42.300434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1186c9-9852-44ae-8898-ac648520a3e9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:42.306503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:42.307552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:42.308036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:42.308587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:42.309441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:42.311318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:42.324767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:42.326064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:42.327002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:42.330463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 360/729] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:42.337836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.372905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:42.373893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:42.374360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:42.374813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] PATH_INFO: `/v3/services/image` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:42.376310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:42.379363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:42.396526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:42.398509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:42.400808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:42.404107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 370/730] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/services/image => generated 86 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:42.409209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.517905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d03ee4-e2a6-4d64-a387-01b79dfa6412 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:42.522578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:42.546059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a88e0-39c8-44ff-a095-f5825ef0722c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:42.565502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:42.566953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:42.567665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:42.568408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:42.569812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:42.570656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:42.573220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:42.589030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:42.589030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:42.589030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:42.589030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:42.589478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] RBAC: Adding query filter params (name=image) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:42.589478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:42.589478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:42.590199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 361/731] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:42.594265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.595674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 371/732] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:42.602401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.683495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:42.717379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137282be-55c8-42d0-9876-a35c00441d66 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:42.755949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:42.756824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:42.757145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:42.757419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:42.758055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] RBAC: Adding query filter params (type=image) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:42.758354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:42.759726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:42.772007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 372/733] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:42.790386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40ac6b-e120-4d4d-9eb6-340268547bde None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:42.850392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:42.851450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:42.852216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:42.852691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:42.853612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:42.854265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:42.855947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:42.869286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 362/734] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:42.871503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:42.882685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:43.007966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b0857-750e-4b7f-b16d-dfdda38bd34f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:43.068892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:43.069614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:43.069893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:43.070162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:43.071191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] RBAC: Adding query filter params (interface=admin, service_id=85bfdf5a1425432198d6078fff6cf332, region_id=RegionOne) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:43.071658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:43.073565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74408d-2f3b-4afb-8bf2-f73c005d8aaa None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:43.081674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 373/735] 10.222.0.30 () {56 vars in 1249 bytes} [Wed Nov 2 14:20:42 2022] GET /identity/v3/endpoints?service_id=85bfdf5a1425432198d6078fff6cf332&interface=admin®ion_id=RegionOne => generated 193 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:43.087584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:43.226337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce1ae0-43ec-4d1c-b7b5-56db08750f31 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:43.240106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:43.267552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541b1cb-9a32-4c0b-8bb4-79226a235da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:43.268515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541b1cb-9a32-4c0b-8bb4-79226a235da3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:43.269030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541b1cb-9a32-4c0b-8bb4-79226a235da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:43.269484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541b1cb-9a32-4c0b-8bb4-79226a235da3 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:43.270342 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541b1cb-9a32-4c0b-8bb4-79226a235da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:43.271939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541b1cb-9a32-4c0b-8bb4-79226a235da3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:43.279160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 363/736] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:43 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:20:43.494045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b68216-16b9-477c-8f84-2fa473b550bc None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:43.494633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b68216-16b9-477c-8f84-2fa473b550bc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:43.494911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b68216-16b9-477c-8f84-2fa473b550bc None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:43.496449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 374/737] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:43.503840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab466-e610-4b97-8d7d-66d1d06472ac None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:43.504491 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab466-e610-4b97-8d7d-66d1d06472ac None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:43.504975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab466-e610-4b97-8d7d-66d1d06472ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:43.526927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-942ab466-e610-4b97-8d7d-66d1d06472ac None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:43.528693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-942ab466-e610-4b97-8d7d-66d1d06472ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:43.618990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 364/738] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:43.927807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b68216-16b9-477c-8f84-2fa473b550bc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:44.015059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:44.053595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b68216-16b9-477c-8f84-2fa473b550bc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:44.114252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bf271-3f5b-4232-a57b-ca3181e8c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:44.115252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bf271-3f5b-4232-a57b-ca3181e8c6a3 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:44.115717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bf271-3f5b-4232-a57b-ca3181e8c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:44.116157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bf271-3f5b-4232-a57b-ca3181e8c6a3 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:44.117353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bf271-3f5b-4232-a57b-ca3181e8c6a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:44.119784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bf271-3f5b-4232-a57b-ca3181e8c6a3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:44.207999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 375/739] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:43 2022] POST /identity/v3/services => generated 242 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:44.241886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408ee14-894c-45f0-81dd-d46c2c7edd1c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:44.242541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408ee14-894c-45f0-81dd-d46c2c7edd1c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:44.242843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408ee14-894c-45f0-81dd-d46c2c7edd1c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:44.245628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 365/740] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:44.255403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:44.255819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:44.256310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:44.288333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:44.290298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:44.393159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 376/741] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:44.810021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408ee14-894c-45f0-81dd-d46c2c7edd1c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:45.024393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408ee14-894c-45f0-81dd-d46c2c7edd1c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:45.029096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:45.081081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:45.082334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.082643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.082923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:45.083591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:45.085086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:45.087994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING keystone.server.flask.common [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 14:20:45.147567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.sql.core [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 14:20:45.147567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 14:20:45.147567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [parameters: {'id': '54eec3229a3d4b84b5e3d3e71bf864b0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 14:20:45.147567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81802) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-999ee3ca-03c7-4b4e-a5cc-8f2b8a85aedb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 54eec3229a3d4b84b5e3d3e71bf864b0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 54eec3229a3d4b84b5e3d3e71bf864b0. Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:20:45.156328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result.read() Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 14:20:45.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application yield resource Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise self.value Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application session.commit() Nov 02 14:20:45.159426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.session.flush() Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application compat.raise_( Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 14:20:45.162012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 14:20:45.164390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application result.read() Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:20:45.166494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [parameters: {'id': '54eec3229a3d4b84b5e3d3e71bf864b0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:45.168701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:45.177932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 54eec3229a3d4b84b5e3d3e71bf864b0. Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:45.178853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 366/742] 10.222.0.30 () {60 vars in 1140 bytes} [Wed Nov 2 14:20:44 2022] POST /identity/v3/projects => generated 262 bytes in 349 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:45.179425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:45.268437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:45.348019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6f6f-fe8f-4406-8961-423713bb07f5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:45.397316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:45.398145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.398481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.398794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:45.407695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:45.409308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:45.423164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:45.424924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:45.426739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 377/743] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:45.435595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad9d9-ec64-41c3-8b8d-19c624be4d40 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.436513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad9d9-ec64-41c3-8b8d-19c624be4d40 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.436513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad9d9-ec64-41c3-8b8d-19c624be4d40 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:45.440433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:45.443090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 367/744] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:45.450016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.450660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.451250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:45.473694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:45.475576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:45.565336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 368/745] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:45.569721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93dc45-39aa-4908-b6bc-9f71451b72e3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:45.592268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:45.615132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:45.615755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.616030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.616306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:45.616995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] RBAC: Adding query filter params (name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:45.617326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:45.618875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539bc78-a934-4328-b093-797d3690d10b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:45.639726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 378/746] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:45.889838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:45.976448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9908ec5-b4f8-4b19-ac54-f0e03bbe62da None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.976734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9908ec5-b4f8-4b19-ac54-f0e03bbe62da None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.977304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9908ec5-b4f8-4b19-ac54-f0e03bbe62da None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:45.979000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 379/747] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:45.984634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:45.986153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:45.986447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:46.008233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:46.011609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:46.013853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:46.025460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfa7fd-7706-4881-8401-e2aedfbd2d26 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:46.072366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:46.072988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:46.073291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:46.073561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:46.075205 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:46.076836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:46.123663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 380/748] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:46.188191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 369/749] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:45 2022] POST /identity/v3/services => generated 242 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:46.465132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:46.616753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ab00b-98b9-48e4-a9d7-26029b3919bf None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:46.638215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:46.667998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:46.668686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:46.668965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:46.669273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:46.669900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:46.671625 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:46.682669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:46.683734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:46.684656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 381/750] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 14:20:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:46.691917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:46.836968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003b8fa-4a43-4a23-a876-af238d70eefb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:46.878027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:46.878670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:46.878944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:46.879217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:46.879856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:46.880157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:46.881579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de43152-747e-4ed7-9d15-2198d5f4a52d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:46.887938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 370/751] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:46.894176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:46.928143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e3378-807c-4001-a394-f5c156a0256f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:46.928418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e3378-807c-4001-a394-f5c156a0256f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:46.928717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e3378-807c-4001-a394-f5c156a0256f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:46.930959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 371/752] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:46.945359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:46.945990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:46.946539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:46.983782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:46.986754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:47.004408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:47.036727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a0123-7baf-4342-9a11-c260e0545e62 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:47.084855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:47.085503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:47.085825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:47.086042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:47.086676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:47.087049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:47.088360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:47.097267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 372/754] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:46 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:47.099072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 382/754] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 14:20:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:47.100715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:47.267131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e2d44-0585-4250-8248-1ccd008c0f0d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:47.317292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:47.317976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:47.318258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:47.318522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:47.319195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:47.320641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:47.373483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:47.410953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:47.415782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 373/755] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:47.535079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1469821-d3aa-40f9-8693-3f44de6cfa1a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:47.590808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:47.592444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:47.592984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:47.593544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:47.595030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:47.596867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:47.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:47.608373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:47.609271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:47.612688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 383/756] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:47.620071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:47.626344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:47.774545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79525bf-70c6-4421-bd4c-0ad07dea3805 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:47.830904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:47.831882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:47.832232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:47.832531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:47.833355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:47.833687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:47.835689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:47.844112 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 374/757] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:47.850475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:47.980725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:47.990843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b839569-da69-4a09-8d16-8647731c56d4 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:48.029708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:48.030327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.030604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.030874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.031783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] RBAC: Adding query filter params (type=compute) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:48.032170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:48.033642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:48.039600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 384/758] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:48.045569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:48.121947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:48.161812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc10139-96a5-497e-b5e6-c30cc3b37aa3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:48.213984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:48.214720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.215059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.215469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.216229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] RBAC: Adding query filter params (interface=public, service_id=c6f8af15da1a43fdb8adac1d02f06842, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:48.216546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:48.218047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b039c-1e09-4ad8-8049-23d274cb3260 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:48.225120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 375/759] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:48 2022] GET /identity/v3/endpoints?service_id=c6f8af15da1a43fdb8adac1d02f06842&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:48.231833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:48.358084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dd398-16c9-46e9-80ed-233915b12181 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.358912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dd398-16c9-46e9-80ed-233915b12181 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.359710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dd398-16c9-46e9-80ed-233915b12181 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.362629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 376/760] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:48.369626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a7acf-1a49-4d71-822e-3b72e30db613 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.370036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a7acf-1a49-4d71-822e-3b72e30db613 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.370404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a7acf-1a49-4d71-822e-3b72e30db613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.377736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015f483-b701-45ec-87b2-35b7e6595eec None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:48.389947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5d6a7acf-1a49-4d71-822e-3b72e30db613 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:48.392008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5d6a7acf-1a49-4d71-822e-3b72e30db613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:48.432202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:48.432937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.433347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.433716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.434545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:48.436416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:48.446981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 385/761] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:48 2022] GET /identity/v3/services => generated 1957 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:20:48.477724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 377/762] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:48 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:48.809679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:48.838990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:48.897511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e335-6c46-4ec8-aae3-43785e5b63a1 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.898045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e335-6c46-4ec8-aae3-43785e5b63a1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.898125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e335-6c46-4ec8-aae3-43785e5b63a1 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.906069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 378/763] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:48.908208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:48.908827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:48.909330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:48.929725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:48.931090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:48.963250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fc0f1-c1ba-4143-b6af-bbfde4a2f469 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:49.002897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:49.012808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:49.013131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 379/764] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:48 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:49.013536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:49.013907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:49.014402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:49.015376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:49.025224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:49.025224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING keystone.server.flask.common [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 14:20:49.084126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.sql.core [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 14:20:49.084126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 14:20:49.084126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [parameters: {'user_id': '9e30ed8558db45e394f914318e677f8f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 14:20:49.084126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81803) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result.read() Nov 02 14:20:49.106134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:20:49.107192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application yield resource Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise self.value Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application session.commit() Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 14:20:49.108148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.session.flush() Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application compat.raise_( Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 14:20:49.109088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:20:49.110065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result.read() Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:20:49.111063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [parameters: {'user_id': '9e30ed8558db45e394f914318e677f8f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:49.111897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 14:20:49.112688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 14:20:49.115532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 14:20:49.115532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 14:20:49.115532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 14:20:49.115532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 14:20:49.115532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 14:20:49.115532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.118078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 386/765] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 14:20:48 2022] POST /identity/v3/users => generated 160 bytes in 309 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:49.124045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:49.314513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3ce11-49c9-4372-a40b-25664de33088 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:49.327488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:49.373323 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:49.374240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:49.374633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:49.375620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:49.387880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:49.390084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:49.410760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:49.412699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 14:20:49.415124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:49.419135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 380/766] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:49.420716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:49.508062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:49.534534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:49.579232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd52b1e-5888-4915-8007-8a6ffc2d17dc None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:49.591263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bc511-fc4f-4418-b587-03e284702710 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:49.639354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:49.640517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:49.641235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:49.641757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:49.642709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:49.644867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:49.658201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:49.658892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:49.660024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:49.660992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:49.661893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:49.661893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:49.661893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:49.661893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:49.661893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:49.670456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:49.671035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 387/767] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Nov 2 14:20:49 2022] GET /identity/v3/services/image => generated 86 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:49.672881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:49.749407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 381/768] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 14:20:49 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:49.821577 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54407-bf78-46ba-a991-fd235d6079f5 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:49.865501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:49.866330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:49.866707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:49.866991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:49.867758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] RBAC: Adding query filter params (name=image) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:49.868116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:49.869993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:49.881297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 388/769] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:49.887900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:50.016344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186220e-a770-4497-a469-46ca51e473ac None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:50.029819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:50.058467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:50.059085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:50.059451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:50.059713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:50.060349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:50.060646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:50.062142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:50.069289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 382/770] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:50.075323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:50.089673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:50.223466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138ab59-3d1d-486a-aab0-ac7ecd77d380 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:50.272200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:50.273130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:50.273551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:50.273960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:50.274895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] RBAC: Adding query filter params (interface=public, service_id=7f37f7aacd524d08a6bb0bf9e13f7ef5, region_id=RegionOne) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:50.275355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:50.276877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05d24a-d37a-45b2-9736-39168f5e417b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:50.283328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 389/771] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:20:50 2022] GET /identity/v3/endpoints?service_id=7f37f7aacd524d08a6bb0bf9e13f7ef5&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:50.290490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:50.326108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198523f-d679-4b02-8a20-33e471510f0e None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:50.326551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198523f-d679-4b02-8a20-33e471510f0e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:50.326985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198523f-d679-4b02-8a20-33e471510f0e None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:50.330241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 390/772] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:50.336693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:50.337156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:50.337563 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:50.360448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:50.362033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:50.442414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 391/773] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:50.476659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a09a-eae6-482d-9554-dcb5e65138f8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:50.525943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:50.526734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:50.527086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:50.527441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:50.528285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:50.530073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:50.541263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 383/774] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:20:50 2022] GET /identity/v3/services => generated 1957 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:20:50.715118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:50.717397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:50.865257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3771f-6877-4bfd-b9b3-2e33ec1f3146 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:50.901414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8df98-af75-4499-a6d8-76a912ce28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:50.934815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8df98-af75-4499-a6d8-76a912ce28ef None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:50.934815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8df98-af75-4499-a6d8-76a912ce28ef None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:50.934815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8df98-af75-4499-a6d8-76a912ce28ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:50.934815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8df98-af75-4499-a6d8-76a912ce28ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:50.934815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8df98-af75-4499-a6d8-76a912ce28ef None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:50.934815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 392/775] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:50.935773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:50.958053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:51.026545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44f68-88a4-4cd4-8381-8ab948cfb617 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:51.065842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:51.066456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:51.066742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:51.066994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:51.067647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:51.069125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:51.143671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198050-afd9-4b03-a4ff-9bda57825b81 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:51.143856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198050-afd9-4b03-a4ff-9bda57825b81 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:51.144219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198050-afd9-4b03-a4ff-9bda57825b81 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:51.145716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 393/776] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:51.150973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:51.151250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:51.151520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:51.171029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:51.173153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:51.224627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 384/777] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:20:50 2022] POST /identity/v3/projects => generated 311 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:51.256055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:51.262204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 394/778] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:51.548984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:51.658945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:51.669384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e8382-abc4-49f2-9d3f-ec0dfde9fadf None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:51.705651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:51.706692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:51.707176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:51.707669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:51.708460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:51.710187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:51.720512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:51.721522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 14:20:51.722470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:51.723206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 385/779] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:51.729557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:51.849197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515ba3-0628-4de1-acc0-e8722240ec61 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:51.895256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:51.896535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:51.896966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:51.897433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:51.898221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] RBAC: Adding query filter params (name=compute) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:51.898730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:51.900428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:51.910528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 395/780] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:51.916275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:52.042567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679b288-3c71-45c6-8f74-855bfce853bf None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:52.080748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:52.081727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:52.082219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:52.082692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:52.083514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:52.084018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:52.085600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481b457-d526-43b6-b888-b4ac6709bd0a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:52.096513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 386/781] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 14:20:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:52.097982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:52.173222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:52.215539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86b761-a8a9-45f7-a240-45426a5b96ba None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:52.378192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebdc2b-5518-44f7-93da-d8099971dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:52.379232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebdc2b-5518-44f7-93da-d8099971dc3e None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:52.379715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebdc2b-5518-44f7-93da-d8099971dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:52.380282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebdc2b-5518-44f7-93da-d8099971dc3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:52.381483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebdc2b-5518-44f7-93da-d8099971dc3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:52.384055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebdc2b-5518-44f7-93da-d8099971dc3e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:52.533875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 396/782] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:52.896858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b703-25d8-4186-8b79-7d77dedd92d9 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:52.897258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b703-25d8-4186-8b79-7d77dedd92d9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:52.897603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b703-25d8-4186-8b79-7d77dedd92d9 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:52.899711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 387/783] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:52.906716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:52.907127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:52.907441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:52.930475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:52.931657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:53.024739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 397/784] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:53.318757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593b703-25d8-4186-8b79-7d77dedd92d9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:53.340895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:53.435873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593b703-25d8-4186-8b79-7d77dedd92d9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:53.478708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:53.479748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:53.480240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:53.480708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] PATH_INFO: `/v3/services/image` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:53.481570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:53.483226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-8fedb8d8-ffb5-42c3-ac92-de4bf1cb2103 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:53.494030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:53.495115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 14:20:53.496038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:53.498996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 388/785] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Nov 2 14:20:53 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:53.509076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:53.589204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:53.589711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875054-6137-41af-b724-a7b3954c7b83 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:53.589890 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875054-6137-41af-b724-a7b3954c7b83 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:53.590121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875054-6137-41af-b724-a7b3954c7b83 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:53.591504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 389/786] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:20:53.597607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:53.597888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:53.598144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:53.631379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:53.635298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:53.688858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc8892-cd42-4bef-a439-31c923c34c0e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:53.744435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:53.745107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:53.745394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:53.745708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:53.746406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] RBAC: Adding query filter params (name=image) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:53.746712 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:53.748098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:53.757050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 398/787] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:53.761214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:53.772741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 390/788] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:20:53.898267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f0628-fd32-4253-acfc-63df6637930f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:53.914518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:53.959568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:53.960633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:53.961174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:53.961666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:53.963074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] RBAC: Adding query filter params (type=image) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:53.963635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:53.966354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:53.976177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 399/789] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:53.986564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:54.005134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:54.091373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:54.113438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700ff61-54c8-46c6-b6e7-d8c7cc54aefd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:54.164518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:54.165203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:54.165651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:54.165981 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:54.166779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:54.168095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:54.228807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7173-5085-41de-aab3-34c36610156b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:54.270840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 391/790] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:20:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:20:54.283419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1b64c-ace0-4f9a-82fc-6a25ece911d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:54.284240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1b64c-ace0-4f9a-82fc-6a25ece911d2 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:54.284597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1b64c-ace0-4f9a-82fc-6a25ece911d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:54.284952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1b64c-ace0-4f9a-82fc-6a25ece911d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:54.286443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1b64c-ace0-4f9a-82fc-6a25ece911d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:54.288238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1b64c-ace0-4f9a-82fc-6a25ece911d2 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:54.293045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 400/791] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:54.297509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:54.388923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:54.449418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f56b-f796-4b78-886e-5bcce2eee206 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:54.493372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6dea-a467-41ce-8ccf-0c073dbcf247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:54.493992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6dea-a467-41ce-8ccf-0c073dbcf247 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:54.494296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6dea-a467-41ce-8ccf-0c073dbcf247 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:54.494565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6dea-a467-41ce-8ccf-0c073dbcf247 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:54.495435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6dea-a467-41ce-8ccf-0c073dbcf247 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:54.496995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6dea-a467-41ce-8ccf-0c073dbcf247 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:54.612657 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 392/792] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:20:54 2022] POST /identity/v3/users => generated 285 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:20:55.249519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136e47-9f8d-4ce1-92a7-db463a80318d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:55.250373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136e47-9f8d-4ce1-92a7-db463a80318d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:55.250373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136e47-9f8d-4ce1-92a7-db463a80318d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:55.252171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 401/793] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:55.261877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:55.262385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:55.262884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:55.303742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:55.306106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:55.448373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 393/794] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:55.919044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39136e47-9f8d-4ce1-92a7-db463a80318d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:56.002215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:56.109762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39136e47-9f8d-4ce1-92a7-db463a80318d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:56.168299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:56.169335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:56.169825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:56.170188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:56.193432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:56.202956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:56.205306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:56.207385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 402/795] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:20:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:56.210616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:56.292612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:56.351616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca8904-8247-433c-a5d6-9303f3647599 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:56.405255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:56.406619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:56.407026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:56.407435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:56.408216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:56.408423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:56.410002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:56.422479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 394/796] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 14:20:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:56.429549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:56.557672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:56.563076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88c9c0-d135-449c-9af2-53060f1583d3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:56.610161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:56.611046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:56.611623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:56.612077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:56.624610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:56.626147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:56.643832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:56.645443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:56.646524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:56.646524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:56.646524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:56.647154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 403/797] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:56.651086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:56.796336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa7592-9cd7-4e3c-a2cf-cbb11eb1e68e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:56.818528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:56.840761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:56.841486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:56.841762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:56.842028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:56.842685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:56.842979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:56.844809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f8139-09bf-47b3-b63d-61d116911df4 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:56.860422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 395/798] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:56.866103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:56.909502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895818-701f-46cb-b273-0370197df2b2 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:56.909502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895818-701f-46cb-b273-0370197df2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:56.909502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895818-701f-46cb-b273-0370197df2b2 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:56.909502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 396/799] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:56.917411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:56.917858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:56.918295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:56.949896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:56.951375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:57.049154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce2039-130d-498e-b88f-a1151e0f7ff0 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:57.049393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 397/800] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:57.097886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:57.098522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.098798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.099066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.108063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:57.110228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:20:57.138019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:57.139312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.141350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 404/801] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 14:20:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:57.149580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:57.349917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0c8fe-11ab-4c5c-afa6-cf49f959015b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:57.364557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:57.376474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:57.393314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:57.416295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:57.417330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.417825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.418289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.419635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:57.420170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:57.423236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:57.463958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 398/802] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 14:20:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:57.465842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:57.539164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4a460-db36-47de-acae-564b765f2c94 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:57.591520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:57.591520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.591862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.592090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.592789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:57.595411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:57.628949 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01fb6-113a-46b9-8eb0-cd39f7136284 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:57.648553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.sql.core [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 14:20:57.648553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 14:20:57.648553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [parameters: {'id': 'a78bb8e8962a4ae492b445438ddc8de6', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 14:20:57.648553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81803) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-fc5b5bcf-20af-4d57-a256-56760050873c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:20:57.657267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result.read() Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 14:20:57.659283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application yield resource Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application next(self.gen) Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise self.value Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application session.commit() Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 14:20:57.661974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.session.flush() Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application compat.raise_( Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 14:20:57.663738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 14:20:57.667775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application conn.query(q) Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application result.read() Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 14:20:57.670147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [parameters: {'id': 'a78bb8e8962a4ae492b445438ddc8de6', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 14:20:57.672835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:57.675633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 405/803] 10.222.0.30 () {60 vars in 1137 bytes} [Wed Nov 2 14:20:57 2022] POST /identity/v3/domains => generated 190 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:57.677967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b929d-384f-4502-ab83-f3ea8f65ee11 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.679766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b929d-384f-4502-ab83-f3ea8f65ee11 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.679766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b929d-384f-4502-ab83-f3ea8f65ee11 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.679766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 406/804] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:20:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:20:57.679766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b929d-384f-4502-ab83-f3ea8f65ee11 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:57.682439 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:57.683494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.683946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.684427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.700823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:57.703546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-6f6024bd-bb80-4650-911f-4a2fa36d74c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:57.722248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:57.723653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:57.724672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:57.724672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:57.724672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:57.724672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 399/805] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:57.725095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.725095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.725369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.751532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:20:57.752773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:20:57.767622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:57.786597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:57.815854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b929d-384f-4502-ab83-f3ea8f65ee11 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:57.834121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 400/806] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:20:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:20:57.834968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:57.852840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:57.853780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:57.854336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:57.854773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:57.866566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:57.868355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:20:57.884802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:57.885816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:20:57.887428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:20:57.887428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:20:57.887428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:57.887428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 407/807] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:20:57.893065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:57.963492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e339c-3bfc-4eb9-9375-c3ca2793add8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.022544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.023334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.023737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.024013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.024788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:58.025392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.027000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.045702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 401/808] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:58.052400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.053124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02b19-28b1-4951-b26d-c9dc3e4c3026 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.068991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:58.085204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:58.094707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.095319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.095595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.095864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.096505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:58.096802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.098247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.113783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 408/809] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:58.121674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.181638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b4ca5-2cfd-419d-b29c-aa4a7ca1dc84 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.232220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.233083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.233392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.233581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.242266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.243892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.263810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0767-af55-4440-9f71-848c24c8286f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:20:58.266385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:58.267390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:20:58.268674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 402/810] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:58.277720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.320952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.321963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.322356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.322742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.344692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:20:58.354315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:20:58.356128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:20:58.357823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 409/811] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:20:58.365425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.389313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:58.441078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5029f-135b-46bb-b86d-80e6d7e07070 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.453966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:58.489268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.489889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.490160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.490427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.491140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:58.491441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.492930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.509104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 403/812] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:58.516833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.532735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4b11-f0ed-49b2-9487-95c11521939f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.573977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.574717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.574851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.575120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.575782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:20:58.576087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.577493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.587304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 410/813] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:20:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:58.593490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.633668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aee57c-db9d-4d66-8cee-5e4af2d9de22 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.661852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:58.670399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.676935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.676935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.676935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/ea17166a0ace485dbf6cadaae7731ed6/roles/a889c76e5a74438a9278ae93b3c0d84a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.676935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=ea17166a0ace485dbf6cadaae7731ed6, role_id=a889c76e5a74438a9278ae93b3c0d84a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.676935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.698875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725b52a-0810-4c05-a460-7bd76748f107 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.713606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:20:58.740186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.740806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.741117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.741391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] PATH_INFO: `/v3/users/617c342106b24329914410654723d389` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.752824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 404/814] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:58 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/ea17166a0ace485dbf6cadaae7731ed6/roles/a889c76e5a74438a9278ae93b3c0d84a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:20:58.756804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] RBAC: Authorizing `identity:get_user(user_id=617c342106b24329914410654723d389)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.758313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.760055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 411/815] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:20:58 2022] GET /identity/v3/users/617c342106b24329914410654723d389 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:20:58.766384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:58.883383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa895e8-a79a-4de1-86e0-14f08b2c77eb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:58.928595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0999413-1337-49da-aa28-d8e72fb6e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:58.929275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999413-1337-49da-aa28-d8e72fb6e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:58.929552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999413-1337-49da-aa28-d8e72fb6e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:58.929913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999413-1337-49da-aa28-d8e72fb6e3c3 None admin] PATH_INFO: `/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:58.930994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0999413-1337-49da-aa28-d8e72fb6e3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=ea74f4a62c1d4bc5813b9c0953691b3f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:58.932453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0999413-1337-49da-aa28-d8e72fb6e3c3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:58.934476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 405/816] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:20:58 2022] GET /identity/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:20:58.940449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:20:59.054629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28338b3c-86b2-4a53-8929-30a60d7e739e None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:20:59.091247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ceece8-71ae-419a-a635-cca4eb54aa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:20:59.091895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ceece8-71ae-419a-a635-cca4eb54aa63 None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:20:59.092170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ceece8-71ae-419a-a635-cca4eb54aa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:20:59.092439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ceece8-71ae-419a-a635-cca4eb54aa63 None admin] PATH_INFO: `/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f/users/617c342106b24329914410654723d389/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:20:59.094550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ceece8-71ae-419a-a635-cca4eb54aa63 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea74f4a62c1d4bc5813b9c0953691b3f, user_id=617c342106b24329914410654723d389, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:20:59.095964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ceece8-71ae-419a-a635-cca4eb54aa63 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:20:59.181676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 412/817] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:20:58 2022] PUT /identity/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f/users/617c342106b24329914410654723d389/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:21:00.955769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872720a-07fb-43a4-86c7-be705499df51 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:00.956416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872720a-07fb-43a4-86c7-be705499df51 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:00.956416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872720a-07fb-43a4-86c7-be705499df51 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:00.957686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 406/818] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:00.963319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a6416-747d-4fa0-badc-2c6e6a616299 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:00.963800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a6416-747d-4fa0-badc-2c6e6a616299 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:00.963891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a6416-747d-4fa0-badc-2c6e6a616299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:00.982875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2e3a6416-747d-4fa0-badc-2c6e6a616299 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:00.984169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2e3a6416-747d-4fa0-badc-2c6e6a616299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:01.009362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:01.059808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 413/819] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:01.332845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872720a-07fb-43a4-86c7-be705499df51 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:01.395159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:01.409477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e14cd-f4fc-43cf-bce0-73eea3c14d4f None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:01.409742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e14cd-f4fc-43cf-bce0-73eea3c14d4f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:01.412250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e14cd-f4fc-43cf-bce0-73eea3c14d4f None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:01.412250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 414/820] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:21:01.417491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:01.417768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:01.418042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:01.437641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:01.438964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:01.446973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872720a-07fb-43a4-86c7-be705499df51 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:01.489279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:01.489920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:01.490193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:01.490467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:01.502870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:21:01.507927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:01.508887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:21:01.509946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 407/821] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 14:21:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:21:01.516587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:01.527212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 415/822] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:21:01.626731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08677e-9708-4dce-ab88-540191ff12af None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:01.641590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:01.662951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:01.663647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:01.663902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:01.664170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:01.664833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:01.670986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:01.670986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:01.676549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 408/823] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 14:21:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:01.682763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:01.788284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:01.804611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90acc72-7912-493f-b57a-2ac0e5a461b5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:01.807200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:01.844236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:01.844864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:01.845169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:01.845439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:01.857489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:01.859735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:21:01.877247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:01.878458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:21:01.879589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:21:01.879589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:21:01.879589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:21:01.879589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 416/824] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:21:01.884319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:01.935854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771915-2bef-492e-b80d-29b71f957009 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:01.974462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:01.975161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:01.975366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:01.975642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:01.987710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:21:01.994293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:01.995469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:21:01.996378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 409/825] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:21:01.999380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9aead-3775-4e59-9e9c-1c42b97f1cc9 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.002681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.111509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.112358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.112988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.113293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.113569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.114260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:02.114566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.115942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.125276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.132685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 417/826] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:02.138035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.184238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550a95b-1e59-4917-8509-cda71ceb9f72 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.228376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.228990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.229296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.229570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.230298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:02.230774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.232203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.242647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 410/827] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:02.249704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.260213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef01e7-613c-4223-afba-fb0a73616a71 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.303212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.304301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.304789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.305312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.320660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.323658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:21:02.347618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:02.349200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:21:02.353189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 418/828] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:21:02.359452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.376782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.380691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842145c3-5fa5-4214-a100-9ab101f400f6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.423854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.424650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.425139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.425588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] PATH_INFO: `/v3/users/617c342106b24329914410654723d389` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.427284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=617c342106b24329914410654723d389)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.428487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.429350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.431666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 411/829] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/users/617c342106b24329914410654723d389 => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:02.439002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.481439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a57ff4-0b78-467e-97e5-03b5fc385634 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.523298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.523911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.524184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.524458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.525260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:02.525549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.526962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.554375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 419/830] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:02.560684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.579897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e0ef3-59d2-4c2d-9ca4-06b01d491fd3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.632121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.632804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.633118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.633407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.676113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.704471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf04932-259e-4d6e-83ad-dc4895b971f1 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:02.723021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.723616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.724521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.747683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:02.748305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:02.748682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:02.748957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:02.843582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 412/831] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 14:21:02.853937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:02.859745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:02.861258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:02.935125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:02.978280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f32bf-85be-454b-b5dc-96afc33f158c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.002803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 420/832] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 14:21:03.008771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:03.029180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:03.038372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.043962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.043962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.043962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] PATH_INFO: `/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.043962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] RBAC: Authorizing `identity:get_project(project_id=ea74f4a62c1d4bc5813b9c0953691b3f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.043962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.044519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 413/833] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:02 2022] GET /identity/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:03.051158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:03.146679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cad98-7104-4f06-af51-eb7f4b742432 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.188663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.189266 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.189544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.189808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.202088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.203578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:21:03.219923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:03.222479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:21:03.225637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:21:03.225637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:21:03.225637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:21:03.225637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 421/834] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:21:03.227293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:03.289274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd91e75-dab1-45db-bc36-ec2e63fa7f54 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.299136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:03.336031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f16f-6b5b-41cd-83dd-a22ed52a9299 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.341747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.342377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.342659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.342999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.344258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=ea74f4a62c1d4bc5813b9c0953691b3f, user.id=617c342106b24329914410654723d389) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:03.344596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.346139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.348253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:03.353580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 414/835] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:21:03 2022] GET /identity/v3/role_assignments?user.id=617c342106b24329914410654723d389&scope.project.id=ea74f4a62c1d4bc5813b9c0953691b3f&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:03.380169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.380792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.381103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.381370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.382040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:03.382341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.383766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.398376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 422/836] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:03.404478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:03.526107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e1768-7d90-4c8e-98a6-4c4401b36acf None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.562425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.563048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.563327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.563583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.570800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.572247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:21:03.585149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:03.586251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:21:03.587379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 415/837] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:21:03.594591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:03.698546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a943c25-0547-41fb-82b5-d589aad029ed None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.727636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:03.736302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.736902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.737203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.737469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.738155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:03.738455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.739985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b563697-ef34-46b4-9a48-2affc4a37710 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.752004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 423/838] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:21:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:03.758227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:03.818494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:03.862202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42003-7dbe-4d3a-b915-d667b838f5ca None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:03.900167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:03.900922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:03.901268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:03.901567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:03.902363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] RBAC: Adding query filter params (role.id=a889c76e5a74438a9278ae93b3c0d84a, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=ea17166a0ace485dbf6cadaae7731ed6) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:03.902740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:03.904288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36a2ff-12eb-45ec-a23c-449b8a383f57 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:03.911042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 416/839] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:21:03 2022] GET /identity/v3/role_assignments?user.id=ea17166a0ace485dbf6cadaae7731ed6&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=a889c76e5a74438a9278ae93b3c0d84a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:05.589091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015efd3-a2dc-42a0-9e06-9fd838fb3d08 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:05.589631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015efd3-a2dc-42a0-9e06-9fd838fb3d08 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:05.589631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015efd3-a2dc-42a0-9e06-9fd838fb3d08 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:05.591060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 424/840] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:05.597241 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:05.597687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:05.597798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:05.616131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:05.617359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:05.683544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 417/841] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:05.955259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015efd3-a2dc-42a0-9e06-9fd838fb3d08 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:06.060680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015efd3-a2dc-42a0-9e06-9fd838fb3d08 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:06.096888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5ef76-d897-43e5-8681-7c2deb9c47ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:06.097530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5ef76-d897-43e5-8681-7c2deb9c47ad None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:06.097808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5ef76-d897-43e5-8681-7c2deb9c47ad None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:06.098077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5ef76-d897-43e5-8681-7c2deb9c47ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:06.099210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5ef76-d897-43e5-8681-7c2deb9c47ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:06.100686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5ef76-d897-43e5-8681-7c2deb9c47ad None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:06.102563 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 425/842] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:21:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:06.108419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:06.164670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:06.212687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956cc19-77e6-4645-b14b-c95211b26688 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:06.246933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce86ff-56c4-42f8-a928-fce1d05c8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:06.247554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce86ff-56c4-42f8-a928-fce1d05c8a45 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:06.247864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce86ff-56c4-42f8-a928-fce1d05c8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:06.248131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce86ff-56c4-42f8-a928-fce1d05c8a45 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:06.248778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce86ff-56c4-42f8-a928-fce1d05c8a45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:06.250850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce86ff-56c4-42f8-a928-fce1d05c8a45 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:06.381907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 418/843] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:21:06 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:21:08.613967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685fa82-0cd0-4b4a-abd0-bffad714446f None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:08.614621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685fa82-0cd0-4b4a-abd0-bffad714446f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:08.614621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685fa82-0cd0-4b4a-abd0-bffad714446f None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:08.615826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 426/844] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:08.621567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:08.622096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:08.622096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:08.641293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:08.642464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:08.711498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 419/845] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:08.987962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685fa82-0cd0-4b4a-abd0-bffad714446f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:09.053506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:09.099329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685fa82-0cd0-4b4a-abd0-bffad714446f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:09.137985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:09.138680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:09.138955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:09.139225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] PATH_INFO: `/v3/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:09.141980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] RBAC: Authorizing `identity:get_role(role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:09.143368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:09.144762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 427/846] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:08 2022] GET /identity/v3/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:09.150717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:09.192305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:09.259712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f232d-a643-4443-a2ff-daed17f26744 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:09.299336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:09.299978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:09.300246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:09.300513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] PATH_INFO: `/v3/users/7850fcb5d1a04ceb9d42b5b42a4274a4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:09.316831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] RBAC: Authorizing `identity:get_user(user_id=7850fcb5d1a04ceb9d42b5b42a4274a4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:09.318344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:09.320050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 420/847] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:09 2022] GET /identity/v3/users/7850fcb5d1a04ceb9d42b5b42a4274a4 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:09.325964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:09.438737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba82fc-eaf2-4a0c-9d36-5f915955f20a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:09.463804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:09.474870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9fbd5-2418-4934-a073-06739f33c874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:09.475489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9fbd5-2418-4934-a073-06739f33c874 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:09.475761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9fbd5-2418-4934-a073-06739f33c874 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:09.476026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9fbd5-2418-4934-a073-06739f33c874 None admin] PATH_INFO: `/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:09.477412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9fbd5-2418-4934-a073-06739f33c874 None admin] RBAC: Authorizing `identity:get_project(project_id=ea74f4a62c1d4bc5813b9c0953691b3f)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:09.478827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9fbd5-2418-4934-a073-06739f33c874 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:09.480433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 428/848] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:09 2022] GET /identity/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:09.486618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:09.582014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:09.588162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf92864-2173-4cf2-a49f-c063bf0ba2cc None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:09.624982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dafeb7-c329-47e2-9e19-44df670a444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:09.625899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dafeb7-c329-47e2-9e19-44df670a444e None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:09.626057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dafeb7-c329-47e2-9e19-44df670a444e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:09.626605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dafeb7-c329-47e2-9e19-44df670a444e None admin] PATH_INFO: `/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f/users/7850fcb5d1a04ceb9d42b5b42a4274a4/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:09.629486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dafeb7-c329-47e2-9e19-44df670a444e None admin] RBAC: Authorizing `identity:create_grant(project_id=ea74f4a62c1d4bc5813b9c0953691b3f, user_id=7850fcb5d1a04ceb9d42b5b42a4274a4, role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:09.630938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dafeb7-c329-47e2-9e19-44df670a444e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:09.752560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 421/849] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:21:09 2022] PUT /identity/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f/users/7850fcb5d1a04ceb9d42b5b42a4274a4/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:21:11.986353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd840119-d68c-4cf0-a5f7-99e05f6637ea None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:11.986904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd840119-d68c-4cf0-a5f7-99e05f6637ea None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:11.986904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd840119-d68c-4cf0-a5f7-99e05f6637ea None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:11.988230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 429/850] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:11.994095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:11.995880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:11.995880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:12.015053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:12.016250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:12.086545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 422/851] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:12.356321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd840119-d68c-4cf0-a5f7-99e05f6637ea None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:12.467405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd840119-d68c-4cf0-a5f7-99e05f6637ea None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:12.503718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:12.504435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:12.504710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:12.504976 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] PATH_INFO: `/v3/roles/22d32c9fcd36484288e27a5f8630fa18` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:12.506395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] RBAC: Authorizing `identity:get_role(role_id=22d32c9fcd36484288e27a5f8630fa18)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:12.507752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:12.509101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 430/852] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:12 2022] GET /identity/v3/roles/22d32c9fcd36484288e27a5f8630fa18 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:21:12.516213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:12.614555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:12.631659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd8276-84c6-4368-9e1b-a69dab889c8f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:12.670342 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:12.671720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:12.671998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:12.672269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] PATH_INFO: `/v3/users/7850fcb5d1a04ceb9d42b5b42a4274a4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:12.673491 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] RBAC: Authorizing `identity:get_user(user_id=7850fcb5d1a04ceb9d42b5b42a4274a4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:12.674980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:12.676619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 423/853] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:12 2022] GET /identity/v3/users/7850fcb5d1a04ceb9d42b5b42a4274a4 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:12.682818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:12.749946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:12.793312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ac81d-bb9f-4f3d-96f9-88b8da8240a3 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:12.829595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:12.830229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:12.830505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:12.830769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:12.914693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:12.916556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:12.979856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:13.052966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 431/854] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:21:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:21:13.060386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:13.184731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0d407-e82e-4cf5-a0a4-5821f518d8b6 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:13.235496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a68c30-75fb-4f1b-ad74-5730c1025ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:13.236273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68c30-75fb-4f1b-ad74-5730c1025ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:13.236555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68c30-75fb-4f1b-ad74-5730c1025ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:13.236822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68c30-75fb-4f1b-ad74-5730c1025ad8 None admin] PATH_INFO: `/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:13.238040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a68c30-75fb-4f1b-ad74-5730c1025ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=ea74f4a62c1d4bc5813b9c0953691b3f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:13.239485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a68c30-75fb-4f1b-ad74-5730c1025ad8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:13.241192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 424/855] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:13 2022] GET /identity/v3/projects/ea74f4a62c1d4bc5813b9c0953691b3f => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:13.247940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:13.402888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a907b-c8f3-4de1-b551-403b43b47d8c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:13.462815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:13.463331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:13.463604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:13.463869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:13.464598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] RBAC: Adding query filter params (role.id=22d32c9fcd36484288e27a5f8630fa18, scope.project.id=ea74f4a62c1d4bc5813b9c0953691b3f, user.id=7850fcb5d1a04ceb9d42b5b42a4274a4) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:13.464906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:13.466365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb378b-8074-49a8-a452-d6496a04315f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:13.472927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 432/856] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:21:13 2022] GET /identity/v3/role_assignments?user.id=7850fcb5d1a04ceb9d42b5b42a4274a4&scope.project.id=ea74f4a62c1d4bc5813b9c0953691b3f&role.id=22d32c9fcd36484288e27a5f8630fa18 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:15.748927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c7c5-d0f4-44b8-976e-63b2ad0e05c5 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:15.749541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c7c5-d0f4-44b8-976e-63b2ad0e05c5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:15.749541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c7c5-d0f4-44b8-976e-63b2ad0e05c5 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:15.750860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 425/857] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:15.756726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:15.757430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:15.757430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:15.776829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:15.778088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:15.798994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:15.856885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 433/858] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:16.135758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384c7c5-d0f4-44b8-976e-63b2ad0e05c5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:16.162019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:16.241445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384c7c5-d0f4-44b8-976e-63b2ad0e05c5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:16.276333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4d46-8a60-40d0-a149-5b685fdf0a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:16.276941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4d46-8a60-40d0-a149-5b685fdf0a02 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:16.277240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4d46-8a60-40d0-a149-5b685fdf0a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:16.277502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4d46-8a60-40d0-a149-5b685fdf0a02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:16.278595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a4d46-8a60-40d0-a149-5b685fdf0a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:16.280038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a4d46-8a60-40d0-a149-5b685fdf0a02 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:16.281656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 426/859] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:21:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:16.287764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:16.389937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5158b-7182-4a74-88dc-6ca29b5ed098 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:16.425674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e90c93-209b-43c2-921a-80d7c5451e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:16.426282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e90c93-209b-43c2-921a-80d7c5451e79 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:16.426557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e90c93-209b-43c2-921a-80d7c5451e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:16.426825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e90c93-209b-43c2-921a-80d7c5451e79 None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:16.427482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e90c93-209b-43c2-921a-80d7c5451e79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:16.428904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e90c93-209b-43c2-921a-80d7c5451e79 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:16.531003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 434/860] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:21:16 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:21:18.785948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3dfa6-d4f5-4b70-957f-08c29c965238 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:18.786514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3dfa6-d4f5-4b70-957f-08c29c965238 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:18.786514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3dfa6-d4f5-4b70-957f-08c29c965238 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:18.787808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 427/861] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:18.793260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:18.793710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:18.793822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:18.807925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:18.814604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:18.815791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:18.883243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 435/862] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:19.157285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3dfa6-d4f5-4b70-957f-08c29c965238 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:19.250992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:19.261984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3dfa6-d4f5-4b70-957f-08c29c965238 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:19.297114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb32ca3-9dd1-4d6f-ac3f-2b147a102d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:19.297737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32ca3-9dd1-4d6f-ac3f-2b147a102d76 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:19.298010 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32ca3-9dd1-4d6f-ac3f-2b147a102d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:19.298273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32ca3-9dd1-4d6f-ac3f-2b147a102d76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:19.299391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb32ca3-9dd1-4d6f-ac3f-2b147a102d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:19.300807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb32ca3-9dd1-4d6f-ac3f-2b147a102d76 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:19.302384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 428/863] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:21:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:19.308271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:19.410152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd480e-09d4-40c1-8724-00c496c53f47 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:19.445710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59048eb1-a9c2-41a9-a449-64e4373957b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:19.446308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59048eb1-a9c2-41a9-a449-64e4373957b8 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:19.446645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59048eb1-a9c2-41a9-a449-64e4373957b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:19.446896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59048eb1-a9c2-41a9-a449-64e4373957b8 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:19.447547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59048eb1-a9c2-41a9-a449-64e4373957b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:19.448974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59048eb1-a9c2-41a9-a449-64e4373957b8 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:19.458353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:19.632278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 436/864] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:21:19 2022] POST /identity/v3/users => generated 286 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:21:21.855443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ee16b-5620-41dc-aa11-c279b1038186 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:21.856011 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ee16b-5620-41dc-aa11-c279b1038186 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:21.856011 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ee16b-5620-41dc-aa11-c279b1038186 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:21.857448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 429/865] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:21.863429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:21.864181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:21.864181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:22.012718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:22.013968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:22.091479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 437/866] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:22.373448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ee16b-5620-41dc-aa11-c279b1038186 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:22.480739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ee16b-5620-41dc-aa11-c279b1038186 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:22.516218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:22.516875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:22.517186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:22.517461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:22.527863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:21:22.533524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:22.534745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:21:22.535809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 430/867] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:21:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:21:22.541147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:22.653103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab32ddf-60fa-49df-bf7b-4112a9dd7805 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:22.677912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:22.690662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:22.695057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:22.695057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:22.695057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:22.695057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:22.695057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:22.695057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:22.705025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 438/868] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:22.710941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:22.732630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:22.822625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e131a-197f-45c3-b758-5199e6f6020c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:22.862906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:22.863716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:22.863816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:22.864046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] PATH_INFO: `/v3/users/6d02b7ce41ad4da9b3a8e4862d737265` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:22.878757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] RBAC: Authorizing `identity:get_user(user_id=6d02b7ce41ad4da9b3a8e4862d737265)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:22.880218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:22.881967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 431/869] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:22 2022] GET /identity/v3/users/6d02b7ce41ad4da9b3a8e4862d737265 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:22.888316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:22.993973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c960-b723-4831-8793-4eed491e5aab None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:23.032154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d510fbb-4a35-4416-8f89-f9d13d261dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:23.032800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510fbb-4a35-4416-8f89-f9d13d261dcb None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:23.033107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510fbb-4a35-4416-8f89-f9d13d261dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:23.033381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510fbb-4a35-4416-8f89-f9d13d261dcb None admin] PATH_INFO: `/v3/projects/2f34a914bf094cc49c8f7f2efe96bc72` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:23.034608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d510fbb-4a35-4416-8f89-f9d13d261dcb None admin] RBAC: Authorizing `identity:get_project(project_id=2f34a914bf094cc49c8f7f2efe96bc72)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:23.036138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d510fbb-4a35-4416-8f89-f9d13d261dcb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:23.037896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 439/870] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:22 2022] GET /identity/v3/projects/2f34a914bf094cc49c8f7f2efe96bc72 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:23.044232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:23.129813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:23.151761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ae5df-26f6-41d1-bb8c-392215a99c8c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:23.188879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8335c1f-733d-4b13-8f1d-f8c4ac23c624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:23.189481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8335c1f-733d-4b13-8f1d-f8c4ac23c624 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:23.189843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8335c1f-733d-4b13-8f1d-f8c4ac23c624 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:23.190119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8335c1f-733d-4b13-8f1d-f8c4ac23c624 None admin] PATH_INFO: `/v3/projects/2f34a914bf094cc49c8f7f2efe96bc72/users/6d02b7ce41ad4da9b3a8e4862d737265/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:23.192005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8335c1f-733d-4b13-8f1d-f8c4ac23c624 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f34a914bf094cc49c8f7f2efe96bc72, user_id=6d02b7ce41ad4da9b3a8e4862d737265, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:23.193465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8335c1f-733d-4b13-8f1d-f8c4ac23c624 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:23.306243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 432/871] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:21:23 2022] PUT /identity/v3/projects/2f34a914bf094cc49c8f7f2efe96bc72/users/6d02b7ce41ad4da9b3a8e4862d737265/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:21:25.498500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08493-070f-47c0-8bf9-ac4dcab963dc None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:25.499072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08493-070f-47c0-8bf9-ac4dcab963dc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:25.499072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08493-070f-47c0-8bf9-ac4dcab963dc None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:25.500383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 440/872] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:25.506172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e481b60-1f06-415b-83bd-818689048cda None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:25.506627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e481b60-1f06-415b-83bd-818689048cda None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:25.506782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e481b60-1f06-415b-83bd-818689048cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:25.524478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4e481b60-1f06-415b-83bd-818689048cda None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:25.525687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4e481b60-1f06-415b-83bd-818689048cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:25.595939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 433/873] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:25.871317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed08493-070f-47c0-8bf9-ac4dcab963dc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:25.927376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:25.975950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed08493-070f-47c0-8bf9-ac4dcab963dc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:26.011499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:26.012121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:26.012400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:26.012672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:26.023397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:21:26.028324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:26.029774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:21:26.030980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 441/874] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:21:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:21:26.035686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e481b60-1f06-415b-83bd-818689048cda None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:26.100340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:26.135260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e481b60-1f06-415b-83bd-818689048cda None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:26.169038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:26.169654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:26.170206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:26.170634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:26.171302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:26.171600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:26.173029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:26.180974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 434/875] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:26.187545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:26.287551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35432fd2-52ea-444e-98b6-986aceab8dc8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:26.300813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:26.322605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:26.323244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:26.323499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:26.323768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] PATH_INFO: `/v3/users/6d02b7ce41ad4da9b3a8e4862d737265` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:26.324880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] RBAC: Authorizing `identity:get_user(user_id=6d02b7ce41ad4da9b3a8e4862d737265)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:26.326481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:26.328682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 442/876] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:26 2022] GET /identity/v3/users/6d02b7ce41ad4da9b3a8e4862d737265 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:26.334597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:26.434388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ded0c-e7c7-4f8b-a027-d3f442704d7b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:26.459622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:26.467746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:26.468340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:26.468615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:26.468879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:26.534841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:26.536275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:26.633500 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 435/877] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:21:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:21:26.640076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:26.738539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f86852-1ef2-458a-a34e-e1c8ca5b3c85 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:26.771408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e38d8-5ea9-4606-96b4-f71d67034cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:26.772032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e38d8-5ea9-4606-96b4-f71d67034cda None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:26.772305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e38d8-5ea9-4606-96b4-f71d67034cda None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:26.772570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e38d8-5ea9-4606-96b4-f71d67034cda None admin] PATH_INFO: `/v3/projects/2f34a914bf094cc49c8f7f2efe96bc72` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:26.773714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e38d8-5ea9-4606-96b4-f71d67034cda None admin] RBAC: Authorizing `identity:get_project(project_id=2f34a914bf094cc49c8f7f2efe96bc72)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:26.775128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e38d8-5ea9-4606-96b4-f71d67034cda None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:26.776737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 443/878] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:26 2022] GET /identity/v3/projects/2f34a914bf094cc49c8f7f2efe96bc72 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:26.783599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:26.810892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:26.884064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352a66-8e1f-4e44-8391-e37b96bf4421 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:26.921884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:26.922495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:26.922768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:26.923038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:26.923761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=2f34a914bf094cc49c8f7f2efe96bc72, user.id=6d02b7ce41ad4da9b3a8e4862d737265) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:26.924065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:26.925556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832444b-1ae0-4fca-a7fa-b089cf7128f7 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:26.931811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 436/879] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:21:26 2022] GET /identity/v3/role_assignments?user.id=6d02b7ce41ad4da9b3a8e4862d737265&scope.project.id=2f34a914bf094cc49c8f7f2efe96bc72&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:29.135121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6b4b-6cda-47ed-9d93-37e955d6925a None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:29.135650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6b4b-6cda-47ed-9d93-37e955d6925a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:29.135650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6b4b-6cda-47ed-9d93-37e955d6925a None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:29.136981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 444/880] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:29.143116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458025-b135-46f5-897b-f1dfcdacef58 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:29.143608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458025-b135-46f5-897b-f1dfcdacef58 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:29.143703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458025-b135-46f5-897b-f1dfcdacef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:29.162033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5d458025-b135-46f5-897b-f1dfcdacef58 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:29.163139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5d458025-b135-46f5-897b-f1dfcdacef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:29.230499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 437/881] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:29.501935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6b4b-6cda-47ed-9d93-37e955d6925a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:29.559619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:29.613996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6b4b-6cda-47ed-9d93-37e955d6925a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:29.652279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c986e35-89d1-4569-af91-4e78c96c9fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:29.652952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c986e35-89d1-4569-af91-4e78c96c9fc4 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:29.656282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c986e35-89d1-4569-af91-4e78c96c9fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:29.656282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c986e35-89d1-4569-af91-4e78c96c9fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:29.656282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c986e35-89d1-4569-af91-4e78c96c9fc4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:29.656282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c986e35-89d1-4569-af91-4e78c96c9fc4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:29.754512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 445/882] 10.222.0.30 () {60 vars in 1138 bytes} [Wed Nov 2 14:21:29 2022] POST /identity/v3/domains => generated 267 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:21:32.115617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e3c16-1096-4cc0-9b24-1bf483171a89 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:32.116158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e3c16-1096-4cc0-9b24-1bf483171a89 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:32.116158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e3c16-1096-4cc0-9b24-1bf483171a89 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:32.117630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 438/883] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:32.124193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:32.124707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:32.124808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:32.144174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:32.145415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:32.215802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 446/884] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:32.489563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e3c16-1096-4cc0-9b24-1bf483171a89 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:32.519173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:32.606637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e3c16-1096-4cc0-9b24-1bf483171a89 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:32.648064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ca005-6587-4cd7-95f8-abe2330c6fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:32.648714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca005-6587-4cd7-95f8-abe2330c6fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:32.648984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca005-6587-4cd7-95f8-abe2330c6fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:32.649282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca005-6587-4cd7-95f8-abe2330c6fc3 None admin] PATH_INFO: `/v3/domains/259a213bed3240babec2ef01ad2a4427` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:32.661660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ca005-6587-4cd7-95f8-abe2330c6fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=259a213bed3240babec2ef01ad2a4427)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:32.663164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ca005-6587-4cd7-95f8-abe2330c6fc3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:32.664901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 439/885] 10.222.0.30 () {56 vars in 1183 bytes} [Wed Nov 2 14:21:32 2022] GET /identity/v3/domains/259a213bed3240babec2ef01ad2a4427 => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:32.672809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:32.717888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:32.787278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90646add-2770-4497-b49a-bbdd958e3ca9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:32.828065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9c7c2-8143-434c-9c01-3b6bed446b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:32.828682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9c7c2-8143-434c-9c01-3b6bed446b90 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:32.828964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9c7c2-8143-434c-9c01-3b6bed446b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:32.829270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9c7c2-8143-434c-9c01-3b6bed446b90 None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:32.829941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9c7c2-8143-434c-9c01-3b6bed446b90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:32.831551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9c7c2-8143-434c-9c01-3b6bed446b90 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:32.997837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 447/886] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:21:32 2022] POST /identity/v3/projects => generated 361 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:21:35.227037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33474538-f2ff-40a9-89b1-a2de29d082db None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:35.227547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33474538-f2ff-40a9-89b1-a2de29d082db None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:35.227547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33474538-f2ff-40a9-89b1-a2de29d082db None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:35.228827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 440/887] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:35.234370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:35.235146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:35.235334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:35.254017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:35.255199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:35.327420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 448/888] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:35.607456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33474538-f2ff-40a9-89b1-a2de29d082db None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:35.690894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:35.710653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33474538-f2ff-40a9-89b1-a2de29d082db None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:35.746508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4849a7-bb9b-4ecc-bbaa-91abdd836d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:35.747129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4849a7-bb9b-4ecc-bbaa-91abdd836d2e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:35.747407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4849a7-bb9b-4ecc-bbaa-91abdd836d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:35.748235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4849a7-bb9b-4ecc-bbaa-91abdd836d2e None admin] PATH_INFO: `/v3/domains/259a213bed3240babec2ef01ad2a4427` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:35.749462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4849a7-bb9b-4ecc-bbaa-91abdd836d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=259a213bed3240babec2ef01ad2a4427)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:35.750899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4849a7-bb9b-4ecc-bbaa-91abdd836d2e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:35.752476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 441/889] 10.222.0.30 () {56 vars in 1183 bytes} [Wed Nov 2 14:21:35 2022] GET /identity/v3/domains/259a213bed3240babec2ef01ad2a4427 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:35.758367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:35.781181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:35.860957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a9455-b3b9-492c-b621-3abcb8bfb9e9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:35.895759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26177914-9b87-44ed-970c-2c89c1cbe5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:35.896367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26177914-9b87-44ed-970c-2c89c1cbe5eb None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:35.896649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26177914-9b87-44ed-970c-2c89c1cbe5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:35.896920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26177914-9b87-44ed-970c-2c89c1cbe5eb None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:35.897640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26177914-9b87-44ed-970c-2c89c1cbe5eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:35.899064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26177914-9b87-44ed-970c-2c89c1cbe5eb None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:35.969400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 449/890] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:21:35 2022] POST /identity/v3/users => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:21:38.193480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343523-b070-40c1-8886-2177534e3e47 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:38.193992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343523-b070-40c1-8886-2177534e3e47 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:38.193992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343523-b070-40c1-8886-2177534e3e47 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:38.195266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 442/891] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:38.201612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:38.201612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:38.202189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:38.220074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:38.221289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:38.288076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 450/892] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:21:38.558028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f343523-b070-40c1-8886-2177534e3e47 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:38.664647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f343523-b070-40c1-8886-2177534e3e47 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:38.701318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:38.701951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:38.702227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:38.702494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:38.714014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:21:38.720037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:38.721200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:21:38.722198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 443/893] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:21:38.728154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:38.737773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:38.827052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef4dad-486f-47c1-ac08-ca5a2c569ad7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:38.862735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:38.863335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:38.863611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:38.863895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:38.864547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:38.864850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:38.866245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:38.875523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 451/894] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:38.881436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:38.894581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:38.983509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5248c-df9e-4159-a2aa-86ae12505689 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:39.017568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:39.018213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:39.018487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:39.018754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] PATH_INFO: `/v3/users/3937c1e1df994e368c4a7061a770a26e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:39.032901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] RBAC: Authorizing `identity:get_user(user_id=3937c1e1df994e368c4a7061a770a26e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:39.034377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:39.036004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 444/895] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:38 2022] GET /identity/v3/users/3937c1e1df994e368c4a7061a770a26e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:39.041860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:39.107950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:39.142285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417d1a7-d75e-40de-a4f4-414ef5ca0d6a None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:39.177380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ae2dc-28fb-4920-91f6-01943f64d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:39.177998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ae2dc-28fb-4920-91f6-01943f64d32d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:39.178270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ae2dc-28fb-4920-91f6-01943f64d32d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:39.178538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ae2dc-28fb-4920-91f6-01943f64d32d None admin] PATH_INFO: `/v3/projects/7953af7f15e24f65b42a0ac070a70961` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:39.179629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ae2dc-28fb-4920-91f6-01943f64d32d None admin] RBAC: Authorizing `identity:get_project(project_id=7953af7f15e24f65b42a0ac070a70961)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:39.181064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ae2dc-28fb-4920-91f6-01943f64d32d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:39.182667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 452/896] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:39 2022] GET /identity/v3/projects/7953af7f15e24f65b42a0ac070a70961 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:39.188728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:39.255629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:39.288064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c6819-062e-442e-96d6-3d90cd36ea22 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:39.323036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c0bb0-1b75-4357-a562-684dd6350a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:39.323689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c0bb0-1b75-4357-a562-684dd6350a69 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:39.323966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c0bb0-1b75-4357-a562-684dd6350a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:39.324240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c0bb0-1b75-4357-a562-684dd6350a69 None admin] PATH_INFO: `/v3/projects/7953af7f15e24f65b42a0ac070a70961/users/3937c1e1df994e368c4a7061a770a26e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:39.326743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c0bb0-1b75-4357-a562-684dd6350a69 None admin] RBAC: Authorizing `identity:create_grant(project_id=7953af7f15e24f65b42a0ac070a70961, user_id=3937c1e1df994e368c4a7061a770a26e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:39.328172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c0bb0-1b75-4357-a562-684dd6350a69 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:39.396791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 445/897] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:21:39 2022] PUT /identity/v3/projects/7953af7f15e24f65b42a0ac070a70961/users/3937c1e1df994e368c4a7061a770a26e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:21:41.568990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198c4f-074b-4363-af06-a50160fc1b7f None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:41.569573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198c4f-074b-4363-af06-a50160fc1b7f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:41.569573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198c4f-074b-4363-af06-a50160fc1b7f None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:41.570926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 453/898] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:21:41.576833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:41.577335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:41.577428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:41.595982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:21:41.597195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:21:41.666994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 446/899] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:21:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:21:41.940643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198c4f-074b-4363-af06-a50160fc1b7f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:42.044405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198c4f-074b-4363-af06-a50160fc1b7f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:42.080246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:42.080900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:42.081203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:42.081477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:42.084122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:42.092563 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:21:42.097626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:21:42.100681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:21:42.101904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 454/900] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:21:42.105265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:42.163792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:42.204780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058329-f753-4caa-9365-16a456b82c38 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:42.238146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:42.238743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:42.239013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:42.239278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:42.239924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:42.240222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:42.241621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:42.251024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 447/901] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:42.257182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:42.355370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2413b14-cde2-4c75-812c-124321b628e8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:42.390248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:42.390871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:42.391147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:42.391418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] PATH_INFO: `/v3/users/3937c1e1df994e368c4a7061a770a26e` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:42.392545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] RBAC: Authorizing `identity:get_user(user_id=3937c1e1df994e368c4a7061a770a26e)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:42.393999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:42.395575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 455/902] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 14:21:42 2022] GET /identity/v3/users/3937c1e1df994e368c4a7061a770a26e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:42.401648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:42.501819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7a35-93e1-4b1c-a95e-90180ea38abb None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:42.521562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:42.535561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:42.536163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:42.536442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:42.536708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:42.607300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:42.608741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:42.707060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 448/903] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:21:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:21:42.713380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:42.776475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:42.815140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b452ecf-54da-4a71-89d4-efc2755bd94b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:42.850369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63df64-03b4-4b5f-94d7-c69bceb27b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:42.850996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63df64-03b4-4b5f-94d7-c69bceb27b1f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:42.851269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63df64-03b4-4b5f-94d7-c69bceb27b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:42.851790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63df64-03b4-4b5f-94d7-c69bceb27b1f None admin] PATH_INFO: `/v3/projects/7953af7f15e24f65b42a0ac070a70961` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:42.852894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63df64-03b4-4b5f-94d7-c69bceb27b1f None admin] RBAC: Authorizing `identity:get_project(project_id=7953af7f15e24f65b42a0ac070a70961)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:42.854401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63df64-03b4-4b5f-94d7-c69bceb27b1f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:42.856014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 456/904] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 14:21:42 2022] GET /identity/v3/projects/7953af7f15e24f65b42a0ac070a70961 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:42.862985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:21:42.883970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:42.963650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aa315-ad5f-4d74-b1e4-d186103cf0a2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:21:43.008169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:21:43.008774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:21:43.009067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:21:43.009330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:21:43.010045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=7953af7f15e24f65b42a0ac070a70961, user.id=3937c1e1df994e368c4a7061a770a26e) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:21:43.010345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:21:43.011775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398878d-a221-455f-97ac-c14599b74b36 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:21:43.018094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 449/905] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:21:42 2022] GET /identity/v3/role_assignments?user.id=3937c1e1df994e368c4a7061a770a26e&scope.project.id=7953af7f15e24f65b42a0ac070a70961&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:21:52.845345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=34 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:21:53.015159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=58 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:03.079591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2744b-04b4-4de3-b2e1-dadbc73d6e05 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:03.080178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2744b-04b4-4de3-b2e1-dadbc73d6e05 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:03.080178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2744b-04b4-4de3-b2e1-dadbc73d6e05 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:03.081504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 457/906] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:03.087841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:03.088339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:03.088440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:03.106478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:03.107647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:03.178951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 450/907] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:03.461421 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2744b-04b4-4de3-b2e1-dadbc73d6e05 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:03.565939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2744b-04b4-4de3-b2e1-dadbc73d6e05 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:03.602556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:03.603175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:03.603452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:03.603849 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:03.614103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:03.628029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:03.632107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:03.633319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:03.635460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:03.635460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:03.635460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:03.635460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 458/908] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:03.640830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:03.742777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe5c74-3598-4f52-b55e-a390f5593e42 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:03.772227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:03.777969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:03.778573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:03.778847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:03.779114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:03.779752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:03.780044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:03.781465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e64fed-001c-4431-b9a8-20e9d47cdbdd None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:03.793120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 451/909] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:03.801161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:03.854743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:03.905360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fa9d9-5c42-44ee-8118-3f90d328360b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:03.940126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d6ab7-59ac-4426-b8c5-22a5ada2bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:03.940719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d6ab7-59ac-4426-b8c5-22a5ada2bbd4 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:03.940995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d6ab7-59ac-4426-b8c5-22a5ada2bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:03.941303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d6ab7-59ac-4426-b8c5-22a5ada2bbd4 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:03.941973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d6ab7-59ac-4426-b8c5-22a5ada2bbd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:03.943392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d6ab7-59ac-4426-b8c5-22a5ada2bbd4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:04.063064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 459/910] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 14:23:03 2022] POST /identity/v3/users => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:23:06.294055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1434d9-ebbc-4b2a-8cc7-6cafa08711c6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:06.294573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1434d9-ebbc-4b2a-8cc7-6cafa08711c6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:06.294573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1434d9-ebbc-4b2a-8cc7-6cafa08711c6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:06.296150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 452/911] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:06.301592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:06.302110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:06.302110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:06.321450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:06.322758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:06.394525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 460/912] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:06.666370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1434d9-ebbc-4b2a-8cc7-6cafa08711c6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:06.769251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1434d9-ebbc-4b2a-8cc7-6cafa08711c6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:06.804087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:06.804710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:06.804983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:06.805282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:06.815197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:23:06.820035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:06.821169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:06.822080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 453/913] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:06.827493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:06.868383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:06.926859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82bc67-01cd-498f-b8cd-1c5e2bc2932b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:06.966775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:06.967408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:06.967684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:06.967952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:06.968620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] RBAC: Adding query filter params (name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:06.968921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:06.970360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:06.979396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 461/914] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:23:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:06.985698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.036914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:07.086695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4579021-ac96-43a1-b03d-54003b335b16 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:07.120199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:07.129739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:07.129739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:07.129739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:07.131194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:07.132876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:07.148829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:07.150065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:07.151036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:07.151036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:07.151036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:07.151036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 454/915] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:07.156286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.253079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:07.256158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8cca-cb8e-4316-a6ed-dec8af97116c None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:07.291766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:07.292373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:07.292653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:07.292930 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:07.293600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:07.293900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:07.295316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:07.306714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 462/916] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:07.312550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.413706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f1eb3-9f9d-458d-80c5-2dc53dbadec0 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:07.415276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:07.447213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:07.447826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:07.448098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:07.448362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:07.455220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:07.456653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:23:07.468553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:07.469783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:07.470767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 455/917] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:07.476771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.577575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fc3d5-a80d-47c7-acf7-0f1492f06595 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:07.611223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:07.611820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:07.612093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:07.612362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:07.613104 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:07.613407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:07.614816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:07.633434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 463/918] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 14:23:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:07.640039 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.748608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685ade-f1c3-4f69-b735-9103ca011981 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:07.785200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:07.785848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:07.786151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:07.786454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:07.796942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:07.798508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:07.812695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:07.813865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:07.815023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:07.815023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:07.815023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:07.815023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 456/919] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:07.820376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.840261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:07.920444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d44b2-e71a-49ad-9f2a-e1396b740e1f None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:07.954573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:07.955178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:07.955457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:07.955726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:07.956369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:07.956740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:07.958210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:07.970512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 464/920] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:07.977543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:07.994987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:08.077085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151a7af-d609-463e-b71c-294b0d96e767 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:08.111429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:08.112044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:08.112314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:08.112574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:08.119338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:08.120776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:23:08.131564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:08.132597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:08.133737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 457/921] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:08.138762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:08.223662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:08.248331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4dc6-f8c9-4be8-a16c-aa72239c361e None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:08.282545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:08.283150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:08.283481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:08.283752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:08.284630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:08.284935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:08.286377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b54d7-66ba-439e-bbbe-306f440f5abc None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:08.299037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 465/922] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:08.305517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:08.380151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:08.405727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8a722-39a8-4d2f-886a-2a5911389d6b None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:08.440458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecd4e5-60db-4661-91cb-5a06371adc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:08.441135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecd4e5-60db-4661-91cb-5a06371adc10 None admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:08.441411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecd4e5-60db-4661-91cb-5a06371adc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:08.441677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecd4e5-60db-4661-91cb-5a06371adc10 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/e7a0aa7593164a57970b48b14c14a41a/roles/2ba31e2aacdc45c6ae755dfba03635e4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:08.456690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecd4e5-60db-4661-91cb-5a06371adc10 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=e7a0aa7593164a57970b48b14c14a41a, role_id=2ba31e2aacdc45c6ae755dfba03635e4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:08.458215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecd4e5-60db-4661-91cb-5a06371adc10 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:08.628118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 458/923] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:23:08 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/e7a0aa7593164a57970b48b14c14a41a/roles/2ba31e2aacdc45c6ae755dfba03635e4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:23:10.819179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b929c7-adea-4fff-bb45-118f8bf2d4b5 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:10.819777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b929c7-adea-4fff-bb45-118f8bf2d4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:10.819900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b929c7-adea-4fff-bb45-118f8bf2d4b5 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:10.821793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 466/924] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:10.827503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:10.827979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:10.838841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:10.846354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:10.847531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:10.918826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 459/925] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:11.185020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b929c7-adea-4fff-bb45-118f8bf2d4b5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:11.288858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b929c7-adea-4fff-bb45-118f8bf2d4b5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:11.322844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:11.323461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:11.323736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:11.324007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:11.330805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:11.334793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:23:11.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:11.344695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:11.345881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 467/926] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 14:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:11.347623 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:11.403362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:11.448499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34666e3b-3a8a-4c32-a241-ab9b72ae25eb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:11.483491 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:11.484111 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:11.484368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:11.484634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:11.485427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] RBAC: Adding query filter params (name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:11.485725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:11.487084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:11.494984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 460/927] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 14:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:11.500967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:11.601846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7bdc-eb39-45ad-b6e0-3e288eca8e56 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:11.635505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:11.636129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:11.636406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:11.636677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:11.648239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:11.649821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:11.665171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:11.666446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:11.667402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:11.667402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:11.667402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:11.667402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 468/928] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:11.672525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:11.778887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52741dd-f380-41a8-a369-370196ada78d None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:11.795528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:11.813696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:11.814292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:11.816632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:11.816632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:11.816632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:11.816632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:11.817152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:11.828929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 461/929] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:11.834870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:11.887897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:11.937767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e673f-6120-418b-9fe9-166801456842 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:11.971638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:11.972261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:11.972541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:11.972810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:11.981355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:11.982801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:23:11.994653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:11.995640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:11.996628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 469/930] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:12.001752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:12.105385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51166c19-87ce-4e0d-8680-90295573d597 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:12.139475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:12.140086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:12.140380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:12.140644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:12.141432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:12.141738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:12.143160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:12.160944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 462/931] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 14:23:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:12.167085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:12.269256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978b745-ec43-455a-86ea-29631a7c49c8 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:12.280561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:12.302739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:12.303334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:12.303612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:12.303882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:12.369428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:12.370749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:12.477106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 470/932] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:23:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:23:12.478556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:12.514558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:12.589477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263b93d-d10b-4608-b3c3-2b6f3b3e8428 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:12.631248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:12.633383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:12.633879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:12.634376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:12.651914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:12.654573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:12.682222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:12.683985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:12.685208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:12.685208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:12.685208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:12.685208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 463/933] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:12.693199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:12.705966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:12.800717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d02b5-453a-47ea-a00b-f9bdce7d0905 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:12.835345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:12.836065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:12.836232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:12.836465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:12.837126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:12.837425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:12.838846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:12.850644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 471/934] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:12.857832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:12.945140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:12.959971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bbd94-7055-4495-9b46-71442e513a15 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:12.995471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:12.996107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:12.996371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:12.996633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:13.003708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:13.005184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:23:13.017287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:13.018407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:13.019780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 464/935] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:13.025040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:13.100203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:13.133407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7235b0e-52a0-4eee-88bd-6a97187fbd28 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:13.167975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:13.168503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:13.168834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:13.169137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:13.169834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:13.170168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:13.171592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036fec-540d-422e-a3b3-41cf567e6b4d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:13.184847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 472/936] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:13.191689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:13.301828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8693a86-61aa-4c10-98a5-878aca1ccfa8 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:13.339975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:13.340606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:13.340810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:13.341118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:13.341842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] RBAC: Adding query filter params (role.id=2ba31e2aacdc45c6ae755dfba03635e4, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=e7a0aa7593164a57970b48b14c14a41a) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:13.342146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:13.343585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13a8c-20b2-4cbe-a170-6fc32c45e21c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:13.349506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 465/937] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:23:13 2022] GET /identity/v3/role_assignments?user.id=e7a0aa7593164a57970b48b14c14a41a&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=2ba31e2aacdc45c6ae755dfba03635e4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:15.555844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf865-94da-4aa8-bb7c-2158afc45b23 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:15.556438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf865-94da-4aa8-bb7c-2158afc45b23 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:15.556438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf865-94da-4aa8-bb7c-2158afc45b23 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:15.557743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 473/938] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:15.563015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:15.563487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:15.563575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:15.583496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:15.585410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:15.587481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:15.654078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 466/939] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:15.931347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf865-94da-4aa8-bb7c-2158afc45b23 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:16.043343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf865-94da-4aa8-bb7c-2158afc45b23 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:16.073594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:16.079403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:16.080027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:16.080309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:16.080580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:16.091436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:23:16.096357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:16.097987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:16.099709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 474/940] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:16.105147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:16.223979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589c20-63da-4c4a-9d07-519aa662b163 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:16.259848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:16.260468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:16.260746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:16.261036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:16.261704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:16.262005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:16.263494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:16.274814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 467/941] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:16.280769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:16.386861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fa49-86f1-4196-aadb-babf0dd73d48 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:16.424164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:16.424389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:16.424569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:16.424847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:16.435524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:16.436971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:16.453213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:16.454410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:16.455397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:16.455397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:16.455397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:16.455397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 475/942] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:16.461667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:16.464579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:16.572833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145de3-f554-40b3-9f32-437566b2208a None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:16.610276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:16.610892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:16.611158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:16.611428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:16.612085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:16.612362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:16.613782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:16.626566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 468/943] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:16.633422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:16.679511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:16.737517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61550367-bc24-4734-a122-23a6b3eab97e None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:16.770191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:16.770819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:16.771095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:16.771366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:16.779222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:16.786033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:23:16.792911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:16.794133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:16.795080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 476/944] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:16.800400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:16.870256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:16.910117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53476f19-c9ce-420a-8604-066d2d7f705e None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:16.950899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:16.951528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:16.951808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:16.952077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:16.952810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:16.953148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:16.954596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:16.973192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 469/945] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 14:23:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:16.980325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:17.082117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de145b-92a6-46c5-ad80-3ff35f0e58b0 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:17.083644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:17.118323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:17.118938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:17.119209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:17.119472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:17.129511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:17.130874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:17.145509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:17.149898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:17.151626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:17.151626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:17.151626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:17.151626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 477/946] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:17.153372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:17.261183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1279d00-9d7d-40e8-85c2-ccd276f5fcc2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:17.282145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:17.296858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:17.297563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:17.297838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:17.298121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:17.298757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:17.299062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:17.300700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:17.313089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 470/947] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:17.318992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:17.427321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b0ed-bbb4-4e19-a4d0-0b62d58ce517 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:17.462649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:17.463263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:17.463537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:17.463805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:17.470348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:17.472987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:23:17.483510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:17.484598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:17.485596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 478/948] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:17.490781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:17.609750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8f06d-466a-460a-89ec-a8a99a323e93 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:17.653515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:17.654154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:17.654478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:17.654752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:17.655479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:17.655770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:17.657262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565ed68-84b7-4314-b9e6-5be92ce82b3a None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:17.671730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 471/949] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:23:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:17.678183 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:17.697646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:17.793912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df2eb-222c-4b6b-bf23-0418a4dd8431 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:17.831791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e01a1-2936-4dc4-a4bf-6dde056d2615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:17.832517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e01a1-2936-4dc4-a4bf-6dde056d2615 None admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:17.832808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e01a1-2936-4dc4-a4bf-6dde056d2615 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:17.833401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e01a1-2936-4dc4-a4bf-6dde056d2615 None admin] PATH_INFO: `/v3/projects/ca7b5445846e410da564b144f3341e91/users/e7a0aa7593164a57970b48b14c14a41a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:17.836608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e01a1-2936-4dc4-a4bf-6dde056d2615 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b5445846e410da564b144f3341e91, user_id=e7a0aa7593164a57970b48b14c14a41a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:17.838280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e01a1-2936-4dc4-a4bf-6dde056d2615 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:17.971419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 479/950] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 14:23:17 2022] PUT /identity/v3/projects/ca7b5445846e410da564b144f3341e91/users/e7a0aa7593164a57970b48b14c14a41a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:23:20.205343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5312-3d3e-4d1a-9a55-403759c67757 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:20.205893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5312-3d3e-4d1a-9a55-403759c67757 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:20.205893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5312-3d3e-4d1a-9a55-403759c67757 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:20.207127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 472/951] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:20.212514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:20.212983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:20.213116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:20.232554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:20.233805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:20.301173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 480/952] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:20.580911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd5312-3d3e-4d1a-9a55-403759c67757 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:20.624270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:20.683238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd5312-3d3e-4d1a-9a55-403759c67757 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:20.717920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:20.718547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:20.718825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:20.719092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:20.730353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 14:23:20.735066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:20.736212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:20.737550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 473/953] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 14:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:20.742575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:20.745916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:20.847369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e081f6-ffa0-4c8e-9e91-a0ae83aa47f1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:20.882706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:20.883323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:20.883616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:20.883916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:20.884615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:20.884925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:20.886495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:20.898870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 481/954] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 14:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:20.905495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:21.015108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:21.018165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04afb66-6f78-478e-9eeb-a7f64b663eb3 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:21.058264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:21.059019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:21.059170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:21.059620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:21.070275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:21.071740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:21.086761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:21.088099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:21.089382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:21.089382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:21.089382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:21.089382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 474/955] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:21.093921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:21.165024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:21.200057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f61c23-43ad-432f-94c8-f61264858f1b None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:21.235290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:21.235991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:21.236153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:21.236418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:21.237099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:21.237405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:21.238935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:21.254531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 482/956] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:21.260453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:21.366008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40938bff-4c6a-4772-af90-adf3453a006c None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:21.401329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:21.401953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:21.402275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:21.402552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] PATH_INFO: `/v3/users/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:21.410303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:21.414616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 14:23:21.423596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:21.424670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:21.425983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 475/957] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:21.430729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:21.536859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e244fcb-3f55-498c-b511-55227abcff00 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:21.563990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:21.573373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:21.573989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:21.574315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:21.574578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:21.575321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:21.575627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:21.577088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:21.596103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 483/958] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 14:23:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:21.602264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:21.618409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:21.705698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bdc6b-b35e-4f6f-9569-bf85c88e2aec None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:21.747695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:21.748315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:21.752316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:21.752316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:21.820902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:21.822392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:21.837564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:21.923178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 476/959] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 14:23:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 14:23:21.930723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:22.044418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8958-ba0a-4914-bbb5-3a07b09e8d76 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:22.079772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:22.080411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:22.080676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:22.080944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:22.091322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:22.092757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 14:23:22.108290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:22.109907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 14:23:22.110802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 14:23:22.110802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 14:23:22.110802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:22.110802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 484/960] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:22.115405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:22.219836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc908d-387a-47c8-800a-487473efd7ee None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:22.233336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:22.254819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:22.255425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:22.255699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:22.255967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:22.257158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:22.257459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:22.258996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:22.270992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 477/961] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 14:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:22.277295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:22.315173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:22.380367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59564e-4ae4-4eaa-932f-f0c3785eb585 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:22.416184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:22.416817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:22.417126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:22.417399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:22.424365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:22.425830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 14:23:22.438453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:22.443172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: ERROR keystone.server.flask.application Nov 02 14:23:22.444671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 485/962] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 14:23:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:22.446268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:22.557614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73af70-ed76-44cb-a6b3-8594ee79e1e2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:22.594324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:22.594831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:22.595108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:22.595374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:22.596067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:22.596363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:22.597835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c057f-95e2-41b0-a9f9-785d5d7fbb7d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:22.610199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 478/963] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 14:23:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:22.617164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:22.715829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:22.721818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f96b0-55b8-45e8-9ba7-332c2393d343 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:22.762047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:22.762646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:22.762923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:22.763195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:22.763942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] RBAC: Adding query filter params (role.id=98256b71f5e74eb59e9c2fbd7b8134b5, scope.project.id=ca7b5445846e410da564b144f3341e91, user.id=e7a0aa7593164a57970b48b14c14a41a) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:22.764250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:22.765706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41d51-c65a-4296-8d80-013bef8b13b4 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:22.771747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 486/964] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 14:23:22 2022] GET /identity/v3/role_assignments?user.id=e7a0aa7593164a57970b48b14c14a41a&scope.project.id=ca7b5445846e410da564b144f3341e91&role.id=98256b71f5e74eb59e9c2fbd7b8134b5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:25.015983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145111-47bd-4497-b37c-021b28a2ebee None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:25.016572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145111-47bd-4497-b37c-021b28a2ebee None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:25.016572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145111-47bd-4497-b37c-021b28a2ebee None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:25.017926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 479/965] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:25.023611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:25.024148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:25.024239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:25.042778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:25.044046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:25.113133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 487/966] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:25.383128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145111-47bd-4497-b37c-021b28a2ebee None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:25.415443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:25.490127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145111-47bd-4497-b37c-021b28a2ebee None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:25.527743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:25.528375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:25.528653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:25.528929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:25.529552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:25.530928 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:25.542152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:25.548925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:25.550053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 480/967] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:23:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:25.551036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:25.644778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:25.658631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb536030-cc32-4cee-a401-0a601542c4c0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:25.694591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:25.695198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:25.695475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:25.695744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:25.696396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:25.696695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:25.698087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905dd6b-829a-4196-b564-1f1fbd977452 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:25.703412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 488/968] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 14:23:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:25.709987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:25.809559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:25.815374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1ccd-0434-4849-8fdd-51add715cd41 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:25.850383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:25.851520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:25.851793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:25.852060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] PATH_INFO: `/v3/services` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:25.852709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:25.853034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:25.854456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe95a-0890-40d9-a8b8-e18a3deec9c3 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:25.859241 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 481/969] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 14:23:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:28.153273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea49be-373d-4ef6-92fa-d5c292e9702a None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:28.153909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea49be-373d-4ef6-92fa-d5c292e9702a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:28.153909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea49be-373d-4ef6-92fa-d5c292e9702a None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:28.155301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 489/970] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:28.161652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e7b9-73fd-47b4-8b2c-8c42a3ee3e33 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:28.162217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e7b9-73fd-47b4-8b2c-8c42a3ee3e33 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:28.162217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e7b9-73fd-47b4-8b2c-8c42a3ee3e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:28.182609 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f186e7b9-73fd-47b4-8b2c-8c42a3ee3e33 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:28.183858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f186e7b9-73fd-47b4-8b2c-8c42a3ee3e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:28.262107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 482/971] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 14:23:28.540116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea49be-373d-4ef6-92fa-d5c292e9702a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:28.664619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea49be-373d-4ef6-92fa-d5c292e9702a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:28.704466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7189d-b629-4c40-9cdd-d3cd24f44a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:28.705116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7189d-b629-4c40-9cdd-d3cd24f44a65 None admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:28.705394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7189d-b629-4c40-9cdd-d3cd24f44a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:28.705682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7189d-b629-4c40-9cdd-d3cd24f44a65 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:28.706453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7189d-b629-4c40-9cdd-d3cd24f44a65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:28.707816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7189d-b629-4c40-9cdd-d3cd24f44a65 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:28.832602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 490/972] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 14:23:28 2022] POST /identity/v3/services => generated 246 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:23:31.100776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef124b-4ad7-4bf9-abe8-272ff4f9989d None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:31.101414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef124b-4ad7-4bf9-abe8-272ff4f9989d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:31.101568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef124b-4ad7-4bf9-abe8-272ff4f9989d None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:31.102923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 483/973] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:23:31.108649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:31.109107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:31.109228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:31.129415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:31.130649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:31.143382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:31.220090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 491/974] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:31.487497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef124b-4ad7-4bf9-abe8-272ff4f9989d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:31.586442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:31.596904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef124b-4ad7-4bf9-abe8-272ff4f9989d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:31.634661 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:31.635292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:31.635571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:31.635837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:31.636413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:31.637819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:31.646786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:31.648808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:31.651023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 484/975] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:23:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 14:23:31.656876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:31.765545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60bf5f-a437-4fa8-adf5-a0ba215f48e8 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:31.807609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:31.808196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:31.808474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:31.808741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:31.809399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:31.809697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:31.811047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:31.816596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 492/976] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 14:23:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:31.823234 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:32.014230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e3138-8c4a-4afd-824e-1b3c23c5a458 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:32.014230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:32.014230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:32.014230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:32.014712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:32.014712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] RBAC: Adding query filter params (interface=public, service_id=356f4257917d4dc0a8c3c1ef47f11ee3, region_id=RegionOne) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:32.014712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:32.014712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be890ffa-84a2-4cf1-870b-bd797c3cfa1d None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:32.014712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 485/977] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 14:23:31 2022] GET /identity/v3/endpoints?service_id=356f4257917d4dc0a8c3c1ef47f11ee3&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:23:32.015114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:32.015114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:32.087671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba9dc-7402-4061-b338-9bc2182b30c1 None admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:32.125155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5bfa0-9ecc-4af6-8054-8d26587a5f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:32.125892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5bfa0-9ecc-4af6-8054-8d26587a5f8d None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:32.126013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5bfa0-9ecc-4af6-8054-8d26587a5f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:32.126275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5bfa0-9ecc-4af6-8054-8d26587a5f8d None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:32.126868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5bfa0-9ecc-4af6-8054-8d26587a5f8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:32.128210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5bfa0-9ecc-4af6-8054-8d26587a5f8d None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:32.135285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 493/978] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 14:23:31 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:23:34.390216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394766b-5f47-4d00-9086-c3bd74fadf90 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:34.390821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394766b-5f47-4d00-9086-c3bd74fadf90 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:34.390821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394766b-5f47-4d00-9086-c3bd74fadf90 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:34.392133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 486/979] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:23:34.397711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:34.398211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:34.398411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:34.417198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:23:34.418417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:23:34.484803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:34.490371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 494/980] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:23:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 14:23:34.766141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394766b-5f47-4d00-9086-c3bd74fadf90 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:34.788425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:34.889117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394766b-5f47-4d00-9086-c3bd74fadf90 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:34.940663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:34.941325 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:34.941593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:34.941859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:34.942437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:34.944383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 14:23:34.953930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 14:23:34.955637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application with Lock( Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self._enter() Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application return self.creator() Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application created_value = creator( Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: ERROR keystone.server.flask.application Nov 02 14:23:34.956743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 487/981] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 14:23:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 14:23:34.962171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:35.079305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761b5b-0b58-4b69-a19b-d8748c8ec331 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:35.118179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:35.118793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:35.119070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:35.119338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] PATH_INFO: `/v3/services` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:35.119969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:23:35.120355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:35.121669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cddbc1-4202-48fb-8223-47b85071d789 None admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:35.128205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 495/982] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 14:23:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:23:35.136520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:23:35.227963 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:35.251198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707beac-32cb-4234-b188-1c0787cc5bc2 None admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:23:35.294230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db15d6-de82-4a07-8c82-b66fdb532a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:23:35.295177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db15d6-de82-4a07-8c82-b66fdb532a29 None admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:23:35.295635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db15d6-de82-4a07-8c82-b66fdb532a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:23:35.296080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db15d6-de82-4a07-8c82-b66fdb532a29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:23:35.297224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db15d6-de82-4a07-8c82-b66fdb532a29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:23:35.299631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db15d6-de82-4a07-8c82-b66fdb532a29 None admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:23:35.480411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 488/983] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:23:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:23:45.123986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=69 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:23:45.469557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:39.864329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587cd7-c0c5-43ca-9cff-e63b693d0583 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:39.865430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587cd7-c0c5-43ca-9cff-e63b693d0583 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:39.865857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587cd7-c0c5-43ca-9cff-e63b693d0583 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:39.875408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913440b4-be3b-4412-ade2-c4f80f03a56d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:39.876468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913440b4-be3b-4412-ade2-c4f80f03a56d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:39.877063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913440b4-be3b-4412-ade2-c4f80f03a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:39.881746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 496/984] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:30:39 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:39.968552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-913440b4-be3b-4412-ade2-c4f80f03a56d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:39.970125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-913440b4-be3b-4412-ade2-c4f80f03a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:40.138006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 489/985] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:40.433177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7d310-0fec-4474-a026-408f461735f5 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:40.433817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7d310-0fec-4474-a026-408f461735f5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:40.434096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7d310-0fec-4474-a026-408f461735f5 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:40.435627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 497/986] 10.222.0.30 () {54 vars in 965 bytes} [Wed Nov 2 14:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:40.444084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725c9ad-c350-4671-b3d6-f50e5d7d7fae None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:40.444623 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725c9ad-c350-4671-b3d6-f50e5d7d7fae None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:40.445248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725c9ad-c350-4671-b3d6-f50e5d7d7fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:40.544220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d725c9ad-c350-4671-b3d6-f50e5d7d7fae None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:40.546785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d725c9ad-c350-4671-b3d6-f50e5d7d7fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:40.591270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:40.683582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 490/987] 10.222.0.30 () {58 vars in 1062 bytes} [Wed Nov 2 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:40.693742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7d310-0fec-4474-a026-408f461735f5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:30:40.696271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-acb7d310-0fec-4474-a026-408f461735f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:40.839023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7d310-0fec-4474-a026-408f461735f5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:30:40.969592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:30:40.971082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:40.971559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:40.972128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:40.974140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:41.044386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:41.107500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:30:41.109103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e532d-712d-4144-a1f6-b1f39a4050f7 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:30:41.130889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 498/988] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:30:43.795791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170f75c-9cba-4e60-b382-43349c95fc8a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:43.796333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170f75c-9cba-4e60-b382-43349c95fc8a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:43.796333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170f75c-9cba-4e60-b382-43349c95fc8a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:43.797742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 491/989] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:43.803544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a302ba-1ffd-4dbd-867c-cea62685d073 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:43.803979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a302ba-1ffd-4dbd-867c-cea62685d073 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:43.804071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a302ba-1ffd-4dbd-867c-cea62685d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:43.927222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-11a302ba-1ffd-4dbd-867c-cea62685d073 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:43.929736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-11a302ba-1ffd-4dbd-867c-cea62685d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:44.057355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 499/990] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:44.350183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170f75c-9cba-4e60-b382-43349c95fc8a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:30:44.351753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e170f75c-9cba-4e60-b382-43349c95fc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:44.472285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170f75c-9cba-4e60-b382-43349c95fc8a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:30:44.556738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:44.559638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:30:44.560680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:44.561193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:44.561677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:44.564365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:44.699895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:30:44.701375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732f411-dead-4e94-81b1-4e85adddb249 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:30:44.721185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 492/991] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:30:47.690156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6006f-6540-44fc-91fa-24c94436b14b None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:47.690746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6006f-6540-44fc-91fa-24c94436b14b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:47.690746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6006f-6540-44fc-91fa-24c94436b14b None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:47.692144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 500/992] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:47.699035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca12ec5-56a0-48f3-bc12-a47a2aa1b40d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:47.699560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca12ec5-56a0-48f3-bc12-a47a2aa1b40d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:47.699560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca12ec5-56a0-48f3-bc12-a47a2aa1b40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:47.770828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cca12ec5-56a0-48f3-bc12-a47a2aa1b40d None None] MFA Rules not processed for user `a7b3a9c09c48450abf3068f29835eb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:47.772101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cca12ec5-56a0-48f3-bc12-a47a2aa1b40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:47.825310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:47.873796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 493/993] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:48.163229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6006f-6540-44fc-91fa-24c94436b14b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:30:48.164830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-28c6006f-6540-44fc-91fa-24c94436b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:48.174888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:48.281723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6006f-6540-44fc-91fa-24c94436b14b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:30:48.363862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:30:48.364472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:48.365314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:48.365584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:48.367436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:48.434686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:48.485443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:30:48.486965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594c9a2-eb72-4f3c-a6ae-5292d9ef69e2 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:30:48.507646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 501/994] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:30:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:30:51.623541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007346-0b21-4fc6-a806-63d4fc97ece4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:51.624068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007346-0b21-4fc6-a806-63d4fc97ece4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:51.624068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007346-0b21-4fc6-a806-63d4fc97ece4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:51.625453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 494/995] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:51.631105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367bb3-bb06-4527-8f40-2d42b77e10f4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:51.631622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367bb3-bb06-4527-8f40-2d42b77e10f4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:51.631622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367bb3-bb06-4527-8f40-2d42b77e10f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:51.700985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4b367bb3-bb06-4527-8f40-2d42b77e10f4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:51.702261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4b367bb3-bb06-4527-8f40-2d42b77e10f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:51.800650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 502/996] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:30:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:52.088902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007346-0b21-4fc6-a806-63d4fc97ece4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:30:52.090447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e0007346-0b21-4fc6-a806-63d4fc97ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:52.199275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007346-0b21-4fc6-a806-63d4fc97ece4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:30:52.278379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:30:52.279381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:52.279811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:52.280157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:52.281931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:52.289667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:52.391798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:30:52.393444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb52f1-9b82-432a-a75c-8f5bc4f887e7 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:30:52.413755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 495/997] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:30:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:30:55.763489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e4916-dd8b-4af5-9822-ad784455c97e None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:55.764078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e4916-dd8b-4af5-9822-ad784455c97e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:55.764078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e4916-dd8b-4af5-9822-ad784455c97e None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:55.765518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 503/998] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:55.770952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c376dd-9502-4343-a93f-6d1fcb754f3c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:55.771396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c376dd-9502-4343-a93f-6d1fcb754f3c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:55.771558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c376dd-9502-4343-a93f-6d1fcb754f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:55.840204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-03c376dd-9502-4343-a93f-6d1fcb754f3c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:55.841437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-03c376dd-9502-4343-a93f-6d1fcb754f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:55.906771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:55.947843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 496/999] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:56.248544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f0e96-649e-42ac-9e0c-d93346eab9b1 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:56.249131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f0e96-649e-42ac-9e0c-d93346eab9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:56.249549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f0e96-649e-42ac-9e0c-d93346eab9b1 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:56.251039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 504/1000] 10.222.0.30 () {54 vars in 965 bytes} [Wed Nov 2 14:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:56.259317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526758d4-779c-4b4a-bd5e-356cc813e071 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:56.259730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526758d4-779c-4b4a-bd5e-356cc813e071 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:56.259934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526758d4-779c-4b4a-bd5e-356cc813e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:56.347196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-526758d4-779c-4b4a-bd5e-356cc813e071 None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:56.348403 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-526758d4-779c-4b4a-bd5e-356cc813e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:56.430238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 497/1001] 10.222.0.30 () {58 vars in 1062 bytes} [Wed Nov 2 14:30:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:56.439310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f0e96-649e-42ac-9e0c-d93346eab9b1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:30:56.440826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bf3f0e96-649e-42ac-9e0c-d93346eab9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:56.460947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:56.547596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f0e96-649e-42ac-9e0c-d93346eab9b1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:30:56.626191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:30:56.626799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:56.627073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:56.627341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:56.628925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:56.706315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:56.746283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:30:56.747844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ba516-ad8f-41f3-9e3a-857dec9a9277 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:30:56.766566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 505/1002] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:30:59.234056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7762-4abd-4ccc-bdb6-b5b45a95f37b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:59.234597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7762-4abd-4ccc-bdb6-b5b45a95f37b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:59.234597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7762-4abd-4ccc-bdb6-b5b45a95f37b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:59.235996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 498/1003] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:30:59.241693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9bf58-9db9-4f99-ac3a-c3876f673d83 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:59.242193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9bf58-9db9-4f99-ac3a-c3876f673d83 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:59.242400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9bf58-9db9-4f99-ac3a-c3876f673d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:59.320025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-74a9bf58-9db9-4f99-ac3a-c3876f673d83 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:30:59.326193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-74a9bf58-9db9-4f99-ac3a-c3876f673d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:59.428556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 506/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:30:59.718909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7762-4abd-4ccc-bdb6-b5b45a95f37b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:30:59.720472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-697e7762-4abd-4ccc-bdb6-b5b45a95f37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:30:59.784195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:30:59.828260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7762-4abd-4ccc-bdb6-b5b45a95f37b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:30:59.912255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:30:59.913585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:30:59.914074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:30:59.914569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:30:59.917783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:00.052544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:00.055700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:00.057357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab12f9-318c-4970-8fb6-c4bf76b8e562 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:00.076310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 499/1005] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:02.401110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42fad-4200-451c-9b93-d17cfc629698 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:02.401630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42fad-4200-451c-9b93-d17cfc629698 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:02.401748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42fad-4200-451c-9b93-d17cfc629698 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:02.403002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 507/1006] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:02.408270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888041f6-57bb-47af-9578-cc68be51a880 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:02.408710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888041f6-57bb-47af-9578-cc68be51a880 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:02.408804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888041f6-57bb-47af-9578-cc68be51a880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:02.480370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-888041f6-57bb-47af-9578-cc68be51a880 None None] MFA Rules not processed for user `a7b3a9c09c48450abf3068f29835eb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:02.482020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-888041f6-57bb-47af-9578-cc68be51a880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:02.591738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 500/1007] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:02.872964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42fad-4200-451c-9b93-d17cfc629698 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:02.874547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-65d42fad-4200-451c-9b93-d17cfc629698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:02.918694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:03.001019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42fad-4200-451c-9b93-d17cfc629698 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:03.085318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:03.085842 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:03.086114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:03.086378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:03.088018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:03.179382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:03.235233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:03.240732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b94f6-e329-4d6a-84dc-4a100fb9f125 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:03.267255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 508/1008] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:06.132336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa2b6d-53ea-4e37-8ee7-97fb618a08bd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:06.132905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa2b6d-53ea-4e37-8ee7-97fb618a08bd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:06.132905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa2b6d-53ea-4e37-8ee7-97fb618a08bd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:06.134246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 501/1009] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:06.139674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b7ba-4c82-4b7c-8a1e-3e1618022d2b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:06.140147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b7ba-4c82-4b7c-8a1e-3e1618022d2b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:06.140270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b7ba-4c82-4b7c-8a1e-3e1618022d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:06.208452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1749b7ba-4c82-4b7c-8a1e-3e1618022d2b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:06.209730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1749b7ba-4c82-4b7c-8a1e-3e1618022d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:06.314131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 509/1010] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:06.608024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa2b6d-53ea-4e37-8ee7-97fb618a08bd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:06.609584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9aaa2b6d-53ea-4e37-8ee7-97fb618a08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:06.682469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:06.716535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa2b6d-53ea-4e37-8ee7-97fb618a08bd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:06.798473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:06.799072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:06.799344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:06.799615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:06.801215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:06.911353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:06.913648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7492c1-6d62-4d84-a067-5c65a4c533b9 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:06.923883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:06.932486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 502/1011] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:09.196884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbb02f-8860-440c-b634-eb723c51674c None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:09.197471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbb02f-8860-440c-b634-eb723c51674c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:09.197471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbb02f-8860-440c-b634-eb723c51674c None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:09.198709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 510/1012] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:09.205136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230fa51-a672-4b0b-81a7-319026bef8ea None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:09.205632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230fa51-a672-4b0b-81a7-319026bef8ea None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:09.206103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230fa51-a672-4b0b-81a7-319026bef8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:09.277831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d230fa51-a672-4b0b-81a7-319026bef8ea None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:09.279050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d230fa51-a672-4b0b-81a7-319026bef8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:09.384499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 503/1013] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:09.668804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbb02f-8860-440c-b634-eb723c51674c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:09.670391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f7fbb02f-8860-440c-b634-eb723c51674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:09.711109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:09.777504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbb02f-8860-440c-b634-eb723c51674c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:09.854814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:09.855670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:09.855958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:09.856230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:09.857869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:09.947665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:09.973531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:09.975075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830cd3e-eede-4d17-8ef2-0cd7b11aa147 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:09.994315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 511/1014] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:12.871323 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a960f-b4e3-4145-af0d-9483bba84e7a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:12.871868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a960f-b4e3-4145-af0d-9483bba84e7a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:12.871868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a960f-b4e3-4145-af0d-9483bba84e7a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:12.873247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 504/1015] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:12.878533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bec2-7d53-49cf-b6a1-b3fb217bb6e7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:12.878948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bec2-7d53-49cf-b6a1-b3fb217bb6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:12.879077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bec2-7d53-49cf-b6a1-b3fb217bb6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:12.954166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bd35bec2-7d53-49cf-b6a1-b3fb217bb6e7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:12.955437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bd35bec2-7d53-49cf-b6a1-b3fb217bb6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:13.057258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 512/1016] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:13.351879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a960f-b4e3-4145-af0d-9483bba84e7a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:13.353553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ea9a960f-b4e3-4145-af0d-9483bba84e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:13.443765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:13.475824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a960f-b4e3-4145-af0d-9483bba84e7a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:13.563105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:13.563731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:13.564005 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:13.564274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:13.566187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:13.705684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:13.707155 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a190-6138-42f3-91ea-3ff113cda1e3 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:13.727611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 505/1017] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:18.702324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17228-fa5e-43b7-90f4-2544d89364d5 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:18.702872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17228-fa5e-43b7-90f4-2544d89364d5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:18.702872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17228-fa5e-43b7-90f4-2544d89364d5 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:18.704490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 513/1018] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:18.711434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d06217-e70b-4b36-b955-06a2c9084f7e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:18.711969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d06217-e70b-4b36-b955-06a2c9084f7e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:18.711969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d06217-e70b-4b36-b955-06a2c9084f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:18.718327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:18.791974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-90d06217-e70b-4b36-b955-06a2c9084f7e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:18.793229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-90d06217-e70b-4b36-b955-06a2c9084f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:18.929838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 506/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:19.219407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a17228-fa5e-43b7-90f4-2544d89364d5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:19.229158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a3a17228-fa5e-43b7-90f4-2544d89364d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:19.274209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:19.332740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a17228-fa5e-43b7-90f4-2544d89364d5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:19.414255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:19.415037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:19.415400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:19.415768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:19.417502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:19.520640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:19.537814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:19.539238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da0142-a04a-4d32-ba34-bca97c5eb57b service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:19.562506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 514/1020] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:22.999728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da851f62-edb0-4ed9-966c-936d2314c931 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:23.000354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da851f62-edb0-4ed9-966c-936d2314c931 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:23.000354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da851f62-edb0-4ed9-966c-936d2314c931 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:23.001731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 507/1021] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:23.007952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e5ba9-e02a-402a-a716-7b48bc3e1728 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:23.008602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e5ba9-e02a-402a-a716-7b48bc3e1728 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:23.008602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e5ba9-e02a-402a-a716-7b48bc3e1728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:23.078437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ce3e5ba9-e02a-402a-a716-7b48bc3e1728 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:23.079618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ce3e5ba9-e02a-402a-a716-7b48bc3e1728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:23.185112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 515/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:23.472496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da851f62-edb0-4ed9-966c-936d2314c931 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:23.474709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-da851f62-edb0-4ed9-966c-936d2314c931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:23.572643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:23.590343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da851f62-edb0-4ed9-966c-936d2314c931 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:23.670989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:23.671603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:23.671947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:23.672208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:23.673848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:23.831765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:23.834395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da23da7-3512-47dd-b05c-5c4c56a740a7 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:23.857692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 508/1023] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:28.669266 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac9e5-ef9f-4524-af79-3b7e0bcdec14 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:28.669772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac9e5-ef9f-4524-af79-3b7e0bcdec14 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:28.669878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac9e5-ef9f-4524-af79-3b7e0bcdec14 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:28.671185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 516/1024] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:28.677350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7491e-21fa-408b-8829-39e87890e07b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:28.677922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7491e-21fa-408b-8829-39e87890e07b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:28.677922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7491e-21fa-408b-8829-39e87890e07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:28.694993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:28.758684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5cf7491e-21fa-408b-8829-39e87890e07b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:28.760886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5cf7491e-21fa-408b-8829-39e87890e07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:28.883065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 509/1025] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:29.164513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ac9e5-ef9f-4524-af79-3b7e0bcdec14 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:29.166194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3a0ac9e5-ef9f-4524-af79-3b7e0bcdec14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:29.232887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:29.287804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ac9e5-ef9f-4524-af79-3b7e0bcdec14 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:29.368519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:29.369162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:29.369443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:29.369710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:29.371365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:29.486344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:29.490681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:29.492161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2afe7-7b73-4ddf-9cd7-0a1365766217 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:29.511074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 517/1026] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:34.704230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1dfc2-b45b-4c4d-baee-d5b1c02f9fe3 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:34.704742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1dfc2-b45b-4c4d-baee-d5b1c02f9fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:34.704870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1dfc2-b45b-4c4d-baee-d5b1c02f9fe3 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:34.706202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 510/1027] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:34.712033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15617-1327-46b7-ae67-6fa185c65c3b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:34.712468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15617-1327-46b7-ae67-6fa185c65c3b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:34.712582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15617-1327-46b7-ae67-6fa185c65c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:34.854487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cad15617-1327-46b7-ae67-6fa185c65c3b None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:34.856300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cad15617-1327-46b7-ae67-6fa185c65c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:35.017905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 518/1028] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:38.881151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=59 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:39.619720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27ffb-85d8-4777-87b1-c7190966619d None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:39.620727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27ffb-85d8-4777-87b1-c7190966619d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:39.621676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27ffb-85d8-4777-87b1-c7190966619d None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:39.623785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 511/1029] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:31:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:39.634105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cab84-414f-46bc-85f4-233435b64740 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:39.634856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cab84-414f-46bc-85f4-233435b64740 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:39.642249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cab84-414f-46bc-85f4-233435b64740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:39.713787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:39.745576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c22cab84-414f-46bc-85f4-233435b64740 None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:39.746795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c22cab84-414f-46bc-85f4-233435b64740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:39.920047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 519/1030] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:42.410585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564edcc6-5d52-4ef8-90ba-c59cd929a730 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:42.411370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564edcc6-5d52-4ef8-90ba-c59cd929a730 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:42.411370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564edcc6-5d52-4ef8-90ba-c59cd929a730 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:42.412561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 512/1031] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:31:42.419397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843eb2c-a1da-4d3d-8b38-26d8a5d96ea0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:42.420192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843eb2c-a1da-4d3d-8b38-26d8a5d96ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:42.420455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843eb2c-a1da-4d3d-8b38-26d8a5d96ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:42.555069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-8843eb2c-a1da-4d3d-8b38-26d8a5d96ea0 None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:42.557198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8843eb2c-a1da-4d3d-8b38-26d8a5d96ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:42.627014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:42.721734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 520/1032] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:31:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:46.174537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42927021-75b6-430c-a004-c16af33ca6a7 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:46.175765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42927021-75b6-430c-a004-c16af33ca6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:46.176420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42927021-75b6-430c-a004-c16af33ca6a7 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:46.182919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 513/1033] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:46.187293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c804-ebac-4cee-a887-dc0d31af6923 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:46.187293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c804-ebac-4cee-a887-dc0d31af6923 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:46.187293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c804-ebac-4cee-a887-dc0d31af6923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:46.315570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-6686c804-ebac-4cee-a887-dc0d31af6923 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:46.317447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6686c804-ebac-4cee-a887-dc0d31af6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:46.467249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 521/1034] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:46.802541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d14a5-9d64-4031-a4e4-cbc8ba50c45c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:46.802759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d14a5-9d64-4031-a4e4-cbc8ba50c45c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:46.802903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d14a5-9d64-4031-a4e4-cbc8ba50c45c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:46.804467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 514/1035] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:46.814734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f138ee-6db6-436d-830d-f6090bdb5029 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:46.815082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f138ee-6db6-436d-830d-f6090bdb5029 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:46.815294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f138ee-6db6-436d-830d-f6090bdb5029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:46.835303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:46.921275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-33f138ee-6db6-436d-830d-f6090bdb5029 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:46.922693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-33f138ee-6db6-436d-830d-f6090bdb5029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:47.075667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 522/1036] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:31:47.085156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d14a5-9d64-4031-a4e4-cbc8ba50c45c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:47.086650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1f8d14a5-9d64-4031-a4e4-cbc8ba50c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:47.254816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d14a5-9d64-4031-a4e4-cbc8ba50c45c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:47.356819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:47.416026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:47.416842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:47.417170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:47.417461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:47.419402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:47.530837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:47.532522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba4d6e-3408-43a8-9841-768e9db37eac service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:47.553260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 515/1037] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:50.100329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25894-6791-4733-b984-9e92ab6bcf2e None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:50.100801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25894-6791-4733-b984-9e92ab6bcf2e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:50.100885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25894-6791-4733-b984-9e92ab6bcf2e None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:50.102614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 523/1038] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:50.108728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033eff-9d86-474f-af74-7937992dcb56 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:50.109327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033eff-9d86-474f-af74-7937992dcb56 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:50.109405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033eff-9d86-474f-af74-7937992dcb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:50.233191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-94033eff-9d86-474f-af74-7937992dcb56 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:50.235078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-94033eff-9d86-474f-af74-7937992dcb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:50.322983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:50.401339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 516/1039] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:50.691674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25894-6791-4733-b984-9e92ab6bcf2e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:50.693387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-13d25894-6791-4733-b984-9e92ab6bcf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:50.781261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:50.897812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25894-6791-4733-b984-9e92ab6bcf2e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:51.058084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:51.058696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:51.058971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:51.059240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:51.060808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:51.097690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:51.180807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:51.182647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856212-9699-4b38-86e1-7feae5051940 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:51.204474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 524/1040] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:54.072611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb587-2a0a-4188-9aeb-2c710547d3c7 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:54.073266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb587-2a0a-4188-9aeb-2c710547d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:54.073266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb587-2a0a-4188-9aeb-2c710547d3c7 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:54.074767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 517/1041] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:31:54.080250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f304b-28ac-45e8-a54c-901ac3a18d6b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:54.080888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f304b-28ac-45e8-a54c-901ac3a18d6b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:54.080888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f304b-28ac-45e8-a54c-901ac3a18d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:54.167357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bd2f304b-28ac-45e8-a54c-901ac3a18d6b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:54.168460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bd2f304b-28ac-45e8-a54c-901ac3a18d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:54.324324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:54.333058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 525/1042] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:54.706755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84412a23-b8bf-4d8e-be7f-26d319431a34 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:54.707073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84412a23-b8bf-4d8e-be7f-26d319431a34 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:54.707441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84412a23-b8bf-4d8e-be7f-26d319431a34 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:54.709215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 518/1043] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 14:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:54.719490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e356176-27c0-4fe6-acc5-7f5b8ab2436d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:54.720199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e356176-27c0-4fe6-acc5-7f5b8ab2436d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:54.720677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e356176-27c0-4fe6-acc5-7f5b8ab2436d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:54.871981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3e356176-27c0-4fe6-acc5-7f5b8ab2436d None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:54.873523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3e356176-27c0-4fe6-acc5-7f5b8ab2436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:55.059898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 526/1044] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:55.074049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84412a23-b8bf-4d8e-be7f-26d319431a34 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:55.076184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-84412a23-b8bf-4d8e-be7f-26d319431a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:55.127931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd82c-a36f-4473-ab32-a8393e394e56 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:55.128802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd82c-a36f-4473-ab32-a8393e394e56 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:55.128802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd82c-a36f-4473-ab32-a8393e394e56 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:55.133459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 527/1045] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:31:55.135537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b45f7-f3b9-4546-b51b-6f75f8fb1ac3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:55.135847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b45f7-f3b9-4546-b51b-6f75f8fb1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:55.136098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b45f7-f3b9-4546-b51b-6f75f8fb1ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:55.231369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:55.245352 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ba2b45f7-f3b9-4546-b51b-6f75f8fb1ac3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:55.247014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ba2b45f7-f3b9-4546-b51b-6f75f8fb1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:55.292794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84412a23-b8bf-4d8e-be7f-26d319431a34 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:55.324852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:55.393075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 528/1046] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:31:55.471541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:55.472941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:55.473571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:55.474007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:55.475946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:55.602919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:55.604916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed9c97-20b9-4a28-9d7b-92937e701f64 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:55.633767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 519/1047] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:55.760593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da3130-a795-4e8d-9632-88366d5a20f0 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:55.760593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da3130-a795-4e8d-9632-88366d5a20f0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:55.760593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da3130-a795-4e8d-9632-88366d5a20f0 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:55.760593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 529/1048] 10.222.0.30 () {54 vars in 963 bytes} [Wed Nov 2 14:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:55.764947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b4c42-d984-40a9-969f-c3025aeffbd6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:55.764947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b4c42-d984-40a9-969f-c3025aeffbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:55.765603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b4c42-d984-40a9-969f-c3025aeffbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:55.803427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:55.894768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-725b4c42-d984-40a9-969f-c3025aeffbd6 None None] MFA Rules not processed for user `20e8f2a80ea142b7acb78ea661b0e47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:55.895876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-725b4c42-d984-40a9-969f-c3025aeffbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:56.004654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 520/1049] 10.222.0.30 () {58 vars in 1060 bytes} [Wed Nov 2 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:31:56.020381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da3130-a795-4e8d-9632-88366d5a20f0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:56.024018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d1da3130-a795-4e8d-9632-88366d5a20f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:56.252284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da3130-a795-4e8d-9632-88366d5a20f0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:56.313572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:56.417340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:56.418325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:56.418841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:56.419152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:56.422451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:56.662461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:56.666490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2ab34-307d-41b3-87c1-0bf6a0708741 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:56.738591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 530/1050] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:31:58.395580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3729dab-663e-44a7-b35b-f2a5b691f8bf None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:58.396391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3729dab-663e-44a7-b35b-f2a5b691f8bf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:58.396391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3729dab-663e-44a7-b35b-f2a5b691f8bf None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:58.397543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 521/1051] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:31:58.403531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7fe35-1a43-4636-8356-a7d2a6a0cd29 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:58.404060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7fe35-1a43-4636-8356-a7d2a6a0cd29 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:58.404191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7fe35-1a43-4636-8356-a7d2a6a0cd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:58.488884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c9f7fe35-1a43-4636-8356-a7d2a6a0cd29 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:31:58.490481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c9f7fe35-1a43-4636-8356-a7d2a6a0cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:58.500204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:58.626134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 531/1052] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:31:58.943055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3729dab-663e-44a7-b35b-f2a5b691f8bf None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:31:58.945430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a3729dab-663e-44a7-b35b-f2a5b691f8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:59.100777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:59.144088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3729dab-663e-44a7-b35b-f2a5b691f8bf None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:31:59.291171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:31:59.291794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:31:59.292078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:31:59.292478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:31:59.294078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:31:59.392967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:31:59.408216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:31:59.410070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef87ca8-f153-42a2-a3c1-2d149b192c76 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:31:59.432384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 522/1053] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:02.330150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cbe3b-da45-4a08-8062-799614d05d22 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:02.330807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cbe3b-da45-4a08-8062-799614d05d22 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:02.331741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cbe3b-da45-4a08-8062-799614d05d22 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:02.334363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 532/1054] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:02.343098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e14228-d46a-4304-a7ca-fca86e165336 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:02.343670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e14228-d46a-4304-a7ca-fca86e165336 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:02.343990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e14228-d46a-4304-a7ca-fca86e165336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:02.446251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-57e14228-d46a-4304-a7ca-fca86e165336 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:02.447848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-57e14228-d46a-4304-a7ca-fca86e165336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:02.573051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 523/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:03.086149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cbe3b-da45-4a08-8062-799614d05d22 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:03.087743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c58cbe3b-da45-4a08-8062-799614d05d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:03.354319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:03.378931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cbe3b-da45-4a08-8062-799614d05d22 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:03.622282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:03.623504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:03.624114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:03.624713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:03.628760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:03.777925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:03.779684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe9e4b-b748-46ea-855f-11f6cdcacb2b service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:03.793819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:03.804750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 533/1056] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:06.623003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d897af-e4e5-4f0a-a6d4-7da7bb34ce41 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:06.623762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d897af-e4e5-4f0a-a6d4-7da7bb34ce41 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:06.623762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d897af-e4e5-4f0a-a6d4-7da7bb34ce41 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:06.625042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 524/1057] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:06.631677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff6bff-9035-4726-a7b8-a5c093aef005 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:06.632260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff6bff-9035-4726-a7b8-a5c093aef005 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:06.632260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff6bff-9035-4726-a7b8-a5c093aef005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:06.713486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a6ff6bff-9035-4726-a7b8-a5c093aef005 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:06.714989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a6ff6bff-9035-4726-a7b8-a5c093aef005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:06.839347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 534/1058] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:07.167905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d897af-e4e5-4f0a-a6d4-7da7bb34ce41 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:07.169771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e9d897af-e4e5-4f0a-a6d4-7da7bb34ce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:07.266616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:07.387444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d897af-e4e5-4f0a-a6d4-7da7bb34ce41 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:07.577026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:07.577915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:07.578284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:07.578715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:07.581845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:07.606569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:07.709904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:07.713180 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b165bd-3724-4bdb-a0fe-c3f69acc5035 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:07.745394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 525/1059] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:10.788683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16886-fe20-48e6-9a73-7ab3ad80e428 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:10.789812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16886-fe20-48e6-9a73-7ab3ad80e428 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:10.789812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16886-fe20-48e6-9a73-7ab3ad80e428 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:10.790971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 535/1060] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:10.797054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76dc2f-dba5-4601-bc12-d3ee2256ea22 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:10.798147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76dc2f-dba5-4601-bc12-d3ee2256ea22 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:10.798147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76dc2f-dba5-4601-bc12-d3ee2256ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:10.878559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fe76dc2f-dba5-4601-bc12-d3ee2256ea22 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:10.879786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fe76dc2f-dba5-4601-bc12-d3ee2256ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:11.005680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:11.017774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 526/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:32:11.338731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e16886-fe20-48e6-9a73-7ab3ad80e428 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:11.340705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-63e16886-fe20-48e6-9a73-7ab3ad80e428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:11.474844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:11.601147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e16886-fe20-48e6-9a73-7ab3ad80e428 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:11.795733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:11.797624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:11.797776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:11.798345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:11.802935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:11.876030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:11.961390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:11.963149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bdf5d-a3ac-442e-a613-516021e143cf service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:11.997831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 536/1062] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:14.404253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3b499-f588-49d1-b8f0-2cde327ac5c3 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:14.405075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3b499-f588-49d1-b8f0-2cde327ac5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:14.405075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3b499-f588-49d1-b8f0-2cde327ac5c3 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:14.408295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 527/1063] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:14.416372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812cbb-8d35-49fb-86f6-dcfde11256fc None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:14.417284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812cbb-8d35-49fb-86f6-dcfde11256fc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:14.418175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812cbb-8d35-49fb-86f6-dcfde11256fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:14.560068 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4d812cbb-8d35-49fb-86f6-dcfde11256fc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:14.566674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4d812cbb-8d35-49fb-86f6-dcfde11256fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:14.703380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 537/1064] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:14.906233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b06a48-22de-4fa9-9903-b1a46336a35b None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:14.906649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b06a48-22de-4fa9-9903-b1a46336a35b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:14.906793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b06a48-22de-4fa9-9903-b1a46336a35b None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:14.908371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 528/1065] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:14.914304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:14.914588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:14.914858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:14.936582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:15.037091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:15.038635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:15.053343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a981e-8f80-4313-b32f-58089451dabf None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:15.053675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a981e-8f80-4313-b32f-58089451dabf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:15.053937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a981e-8f80-4313-b32f-58089451dabf None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:15.055489 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 529/1066] 10.222.0.30 () {54 vars in 962 bytes} [Wed Nov 2 14:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:32:15.061916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:15.062239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:15.062461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:15.187490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] MFA Rules not processed for user `287fbe09913b454c9fcd58febbca6b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:15.190180 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:15.193451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 538/1067] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:15.342154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 530/1068] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Nov 2 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:32:15.355979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:15.358230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:15.543512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:15.545745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:15.547019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61f2ed-273c-4958-9b1b-026647852e3a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:15.557671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:15.655802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:15.656845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:15.657350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:15.657814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:15.660815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:15.696017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:15.772382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06f6eb-f069-4565-a7e7-c297813006c5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:15.871150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:15.872652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c588ab-a9c5-4f53-a77e-366087d6d2d4 service glance] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:15.903354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 539/1069] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:15.974112 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:15.974735 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:15.975013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:15.975280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:15.976978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:16.083271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:16.099806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:16.101462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57487566-68cd-401f-84fd-d92d7553cabf service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:16.125983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 531/1070] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:17.319411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fa33d-632e-4637-8405-4e8074dbb728 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:17.320036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fa33d-632e-4637-8405-4e8074dbb728 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:17.320814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fa33d-632e-4637-8405-4e8074dbb728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:17.414250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:17.444541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-962fa33d-632e-4637-8405-4e8074dbb728 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:17.446864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-962fa33d-632e-4637-8405-4e8074dbb728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:17.610453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 540/1071] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:17.680801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52524-65c3-4153-999a-1c70bc407850 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:17.681413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52524-65c3-4153-999a-1c70bc407850 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:17.681529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52524-65c3-4153-999a-1c70bc407850 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:17.683049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 532/1072] 10.222.0.30 () {54 vars in 943 bytes} [Wed Nov 2 14:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:17.690203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b9e-abd5-4f1d-aba9-e92850feecb8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:17.690490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b9e-abd5-4f1d-aba9-e92850feecb8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:17.690770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b9e-abd5-4f1d-aba9-e92850feecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:17.796046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bcb12b9e-abd5-4f1d-aba9-e92850feecb8 None None] MFA Rules not processed for user `0435bede480b4f83aee82223ef80836f`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:17.797561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bcb12b9e-abd5-4f1d-aba9-e92850feecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:17.884288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:17.956722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 541/1073] 10.222.0.30 () {58 vars in 1040 bytes} [Wed Nov 2 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:17.966866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52524-65c3-4153-999a-1c70bc407850 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:17.968407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e2c52524-65c3-4153-999a-1c70bc407850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:18.136280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52524-65c3-4153-999a-1c70bc407850 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:18.224555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:18.275953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:18.277189 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:18.277465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:18.277736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:18.280564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:18.404094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:18.405824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0dc45-c132-48b8-ad5e-195bb29697cb service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:18.416113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 533/1074] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:32:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 14:32:18.736826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221530b1-a4d8-4f91-b2c8-ab98b08b1264 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:18.737869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221530b1-a4d8-4f91-b2c8-ab98b08b1264 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:18.737869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221530b1-a4d8-4f91-b2c8-ab98b08b1264 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:18.739455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 542/1075] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:18.746056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9d7d1-3808-4525-93bd-fe5672420617 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:18.746896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9d7d1-3808-4525-93bd-fe5672420617 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:18.747868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9d7d1-3808-4525-93bd-fe5672420617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:18.843426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6fa9d7d1-3808-4525-93bd-fe5672420617 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:18.845482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6fa9d7d1-3808-4525-93bd-fe5672420617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:18.966461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:19.056462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 534/1076] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:19.605163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221530b1-a4d8-4f91-b2c8-ab98b08b1264 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:19.608960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-221530b1-a4d8-4f91-b2c8-ab98b08b1264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:19.978075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221530b1-a4d8-4f91-b2c8-ab98b08b1264 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:20.082653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:20.248620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923848af-3e44-431e-8207-3136002470a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:20.249657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923848af-3e44-431e-8207-3136002470a8 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:20.249927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923848af-3e44-431e-8207-3136002470a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:20.250572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923848af-3e44-431e-8207-3136002470a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:20.253467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-923848af-3e44-431e-8207-3136002470a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:20.433636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923848af-3e44-431e-8207-3136002470a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:20.436040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923848af-3e44-431e-8207-3136002470a8 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:20.485670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 543/1077] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 881 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:23.528411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112403a0-57fd-409a-a651-f772e53d006f None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:23.529400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112403a0-57fd-409a-a651-f772e53d006f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:23.529400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112403a0-57fd-409a-a651-f772e53d006f None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:23.530906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 535/1078] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:32:23.539327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199111-f28a-48fe-b858-1b82cc3c3a3a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:23.540234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199111-f28a-48fe-b858-1b82cc3c3a3a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:23.540438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199111-f28a-48fe-b858-1b82cc3c3a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:23.677107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:23.691879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-db199111-f28a-48fe-b858-1b82cc3c3a3a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:23.696050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-db199111-f28a-48fe-b858-1b82cc3c3a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:23.844294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 544/1079] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:24.395567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112403a0-57fd-409a-a651-f772e53d006f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:24.397701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-112403a0-57fd-409a-a651-f772e53d006f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:24.693787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112403a0-57fd-409a-a651-f772e53d006f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:24.777596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:24.940051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:24.940938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:24.941329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:24.941693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:24.943469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:25.090493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:25.092308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338d305-c6cb-44dd-a15a-b71df8c5c665 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:25.116271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 536/1080] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:28.035358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce97ec-d72d-42ff-9223-16a77a54ae4d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:28.035917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce97ec-d72d-42ff-9223-16a77a54ae4d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:28.035917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce97ec-d72d-42ff-9223-16a77a54ae4d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:28.038084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 545/1081] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:28.044811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79f6a-5077-44fc-855c-f4d3e6dd6a3c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:28.045443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79f6a-5077-44fc-855c-f4d3e6dd6a3c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:28.046019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79f6a-5077-44fc-855c-f4d3e6dd6a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:28.120157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:28.145887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d6e79f6a-5077-44fc-855c-f4d3e6dd6a3c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:28.147157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d6e79f6a-5077-44fc-855c-f4d3e6dd6a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:28.311498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 537/1082] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:28.743398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce97ec-d72d-42ff-9223-16a77a54ae4d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:28.746788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0fce97ec-d72d-42ff-9223-16a77a54ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:29.004575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:29.090192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce97ec-d72d-42ff-9223-16a77a54ae4d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:29.434128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:29.435305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:29.435833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:29.436302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:29.440426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:29.617384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:29.626362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:29.629990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a6ef8-1dfd-4157-9a27-e72d8a778017 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:29.663894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 546/1083] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 921 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:33.203090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac125ccf-5e9d-4a51-b267-fee0ec68d2bf None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:33.203744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac125ccf-5e9d-4a51-b267-fee0ec68d2bf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:33.203744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac125ccf-5e9d-4a51-b267-fee0ec68d2bf None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:33.206032 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 538/1084] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:32:33.212079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff5857-e240-4be9-a5d8-56e1281e67ba None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:33.213527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff5857-e240-4be9-a5d8-56e1281e67ba None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:33.213527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff5857-e240-4be9-a5d8-56e1281e67ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:33.306087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-16ff5857-e240-4be9-a5d8-56e1281e67ba None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:33.307777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-16ff5857-e240-4be9-a5d8-56e1281e67ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:33.450583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 547/1085] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:33.774828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac125ccf-5e9d-4a51-b267-fee0ec68d2bf None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:33.781054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ac125ccf-5e9d-4a51-b267-fee0ec68d2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:33.948582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:34.020042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac125ccf-5e9d-4a51-b267-fee0ec68d2bf None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:34.191761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:34.192402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:34.192711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:34.193040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:34.194754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:34.323207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:34.343171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:34.347213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230e1f5-7bc6-474e-b80e-675a537b7b29 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:34.384311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 539/1086] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:43.448600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=72 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:44.380734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=9 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:47.851812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1f8c7-27c1-427f-a171-4f43bd4baa6d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:47.852735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1f8c7-27c1-427f-a171-4f43bd4baa6d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:47.852735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1f8c7-27c1-427f-a171-4f43bd4baa6d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:47.854534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 548/1087] 10.222.0.30 () {54 vars in 918 bytes} [Wed Nov 2 14:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:47.861695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957cf91-fd7c-487e-9f6d-98138b764514 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:47.862280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957cf91-fd7c-487e-9f6d-98138b764514 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:47.862280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957cf91-fd7c-487e-9f6d-98138b764514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:47.954814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5957cf91-fd7c-487e-9f6d-98138b764514 None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:47.956066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5957cf91-fd7c-487e-9f6d-98138b764514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:48.096043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 540/1088] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Nov 2 14:32:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:48.112201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d232f2-3c30-41c7-ae78-cfa1bf9942e2 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:48.112480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d232f2-3c30-41c7-ae78-cfa1bf9942e2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:48.112756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d232f2-3c30-41c7-ae78-cfa1bf9942e2 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:48.114184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 549/1089] 10.222.0.30 () {54 vars in 918 bytes} [Wed Nov 2 14:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:48.119347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b166-212c-4dd7-a61a-f456e4ee4b15 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:48.119627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b166-212c-4dd7-a61a-f456e4ee4b15 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:48.119898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b166-212c-4dd7-a61a-f456e4ee4b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:48.227967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:48.232578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8bb4b166-212c-4dd7-a61a-f456e4ee4b15 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:48.234629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8bb4b166-212c-4dd7-a61a-f456e4ee4b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:48.382767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 541/1090] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Nov 2 14:32:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:51.302546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1e1d-ce5c-4329-9815-95c4ac5048d8 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:51.303243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1e1d-ce5c-4329-9815-95c4ac5048d8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:51.303243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1e1d-ce5c-4329-9815-95c4ac5048d8 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:51.304668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 550/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:51.311775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53e836-443a-4592-9815-b93a5de452b2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:51.312322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53e836-443a-4592-9815-b93a5de452b2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:51.312750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53e836-443a-4592-9815-b93a5de452b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:51.437089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dd53e836-443a-4592-9815-b93a5de452b2 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:51.440143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd53e836-443a-4592-9815-b93a5de452b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:51.508844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:51.560035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 542/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:51.886272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce1e1d-ce5c-4329-9815-95c4ac5048d8 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:51.891989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-91ce1e1d-ce5c-4329-9815-95c4ac5048d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:52.059998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce1e1d-ce5c-4329-9815-95c4ac5048d8 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:52.144552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:52.202343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:52.203343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:52.204460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:52.204929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:52.207912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:52.331069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:52.332755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958877b0-f90e-4041-b36c-8b85f9c89844 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:52.354774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 551/1093] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:53.596523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3487cc-2049-4b73-bf3d-66c0067395ee None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:53.597341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3487cc-2049-4b73-bf3d-66c0067395ee None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:53.597341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3487cc-2049-4b73-bf3d-66c0067395ee None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:53.598910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 543/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:53.605054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c13b0-d07f-4482-929d-528d471ea663 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:53.606080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c13b0-d07f-4482-929d-528d471ea663 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:53.606080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c13b0-d07f-4482-929d-528d471ea663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:53.701261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3c2c13b0-d07f-4482-929d-528d471ea663 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:53.702600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3c2c13b0-d07f-4482-929d-528d471ea663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:53.736171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:53.817277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 552/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:53.988054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3487cc-2049-4b73-bf3d-66c0067395ee None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:53.989783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ad3487cc-2049-4b73-bf3d-66c0067395ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:54.164305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3487cc-2049-4b73-bf3d-66c0067395ee None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:54.213768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:54.303797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:54.304425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:54.304700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:54.304969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:54.306720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:54.412267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:54.413904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff679f83-3fa1-4c42-b793-22153b78d27f service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:54.434766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 544/1096] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:55.619744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb7f58-9b07-443b-baf7-3101c85fb00d None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:55.620517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb7f58-9b07-443b-baf7-3101c85fb00d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:55.620517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb7f58-9b07-443b-baf7-3101c85fb00d None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:55.621744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 553/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:32:55.628107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415d115-f045-4b65-80be-fc7ca72b52ac None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:55.628680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415d115-f045-4b65-80be-fc7ca72b52ac None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:55.628959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415d115-f045-4b65-80be-fc7ca72b52ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:55.710081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:55.737130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b415d115-f045-4b65-80be-fc7ca72b52ac None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:55.738468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b415d115-f045-4b65-80be-fc7ca72b52ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:55.871190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 545/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:56.034303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb7f58-9b07-443b-baf7-3101c85fb00d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:56.035958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e7bb7f58-9b07-443b-baf7-3101c85fb00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:56.306181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb7f58-9b07-443b-baf7-3101c85fb00d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:56.320430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:56.439811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:56.440427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:56.440740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:56.441028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:56.442722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:56.546818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:56.549672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92967664-d9b1-48fc-9ea2-6e7d2cc41722 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:56.570687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 554/1099] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:32:58.044652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6fb94-09ea-4d4a-81e1-7af132cc5f61 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:58.045286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6fb94-09ea-4d4a-81e1-7af132cc5f61 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:58.045286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6fb94-09ea-4d4a-81e1-7af132cc5f61 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:58.046566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 546/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:58.052450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45b311-4815-4a73-959e-de81ba69bca3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:58.053048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45b311-4815-4a73-959e-de81ba69bca3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:58.053196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45b311-4815-4a73-959e-de81ba69bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:58.074833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:58.153737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4c45b311-4815-4a73-959e-de81ba69bca3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:32:58.155356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4c45b311-4815-4a73-959e-de81ba69bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:58.276325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 555/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:32:58.430353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6fb94-09ea-4d4a-81e1-7af132cc5f61 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:32:58.431927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f1b6fb94-09ea-4d4a-81e1-7af132cc5f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:58.558366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:58.598093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6fb94-09ea-4d4a-81e1-7af132cc5f61 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:32:58.728088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:32:58.728693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:58.728966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:58.729262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:58.731167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:32:58.813409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:32:58.826788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:32:58.829553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666953a6-a4c3-4bc3-baff-874449345e17 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:32:58.849950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 547/1102] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:32:59.989452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa42bc-9427-4052-bb95-fab017a59410 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:59.990023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa42bc-9427-4052-bb95-fab017a59410 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:59.990023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa42bc-9427-4052-bb95-fab017a59410 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:32:59.991332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 556/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:32:59.996529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87b0d0-006d-4d4d-aa14-d50368dda1d6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:32:59.996995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87b0d0-006d-4d4d-aa14-d50368dda1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:32:59.997134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87b0d0-006d-4d4d-aa14-d50368dda1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:00.081997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ba87b0d0-006d-4d4d-aa14-d50368dda1d6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:00.083208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ba87b0d0-006d-4d4d-aa14-d50368dda1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:00.189536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 548/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:33:00.343721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa42bc-9427-4052-bb95-fab017a59410 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:00.345397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-25fa42bc-9427-4052-bb95-fab017a59410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:00.442858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:00.510956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa42bc-9427-4052-bb95-fab017a59410 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:00.693499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db939d-0253-45ee-9633-05403516da32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:00.694416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db939d-0253-45ee-9633-05403516da32 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:00.694900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db939d-0253-45ee-9633-05403516da32 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:00.695362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db939d-0253-45ee-9633-05403516da32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:00.698217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-28db939d-0253-45ee-9633-05403516da32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:00.759633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:00.805550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db939d-0253-45ee-9633-05403516da32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:00.807154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db939d-0253-45ee-9633-05403516da32 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:00.827156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 557/1105] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:33:01.965583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394f18a-6064-495c-b320-4282ce15943c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:01.966186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394f18a-6064-495c-b320-4282ce15943c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:01.966186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394f18a-6064-495c-b320-4282ce15943c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:01.967653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 549/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:33:01.974016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ca216-f4ea-40ec-8fea-ec1ad37803d5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:01.974757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ca216-f4ea-40ec-8fea-ec1ad37803d5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:01.974757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ca216-f4ea-40ec-8fea-ec1ad37803d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:02.059335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4e9ca216-f4ea-40ec-8fea-ec1ad37803d5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:02.060573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4e9ca216-f4ea-40ec-8fea-ec1ad37803d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:02.179879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 558/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:33:02.333876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394f18a-6064-495c-b320-4282ce15943c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:02.335593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8394f18a-6064-495c-b320-4282ce15943c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:02.405285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:02.564348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394f18a-6064-495c-b320-4282ce15943c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:02.706515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:02.707202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:02.707526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:02.707841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:02.709736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:02.718242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:02.830055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:02.832775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e84d2-c2c2-41f0-a039-312d8fde6cd6 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:02.861779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 550/1108] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:33:04.042165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76779919-9d0a-4acc-8c19-22fd9620242c None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:04.043035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76779919-9d0a-4acc-8c19-22fd9620242c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:04.043035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76779919-9d0a-4acc-8c19-22fd9620242c None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:04.044699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 559/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:33:04.051170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530810b0-fe30-41e5-883e-f98ce069a965 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:04.051810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530810b0-fe30-41e5-883e-f98ce069a965 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:04.051810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530810b0-fe30-41e5-883e-f98ce069a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:04.143943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-530810b0-fe30-41e5-883e-f98ce069a965 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:04.145232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-530810b0-fe30-41e5-883e-f98ce069a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:04.231563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:04.269570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 551/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:04.427772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76779919-9d0a-4acc-8c19-22fd9620242c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:04.429504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-76779919-9d0a-4acc-8c19-22fd9620242c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:04.449555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:04.600330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76779919-9d0a-4acc-8c19-22fd9620242c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:04.705487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:04.735365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:04.735982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:04.736256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:04.736533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:04.738237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:04.851787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:04.853467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019db4ca-510f-4440-bbae-094815a55154 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:04.874494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 560/1111] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:06.038826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2844993-21ed-4954-9c6d-5f9df2354d74 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:06.039438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2844993-21ed-4954-9c6d-5f9df2354d74 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:06.039577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2844993-21ed-4954-9c6d-5f9df2354d74 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:06.041063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 552/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:06.047210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc827-cb5d-488d-8285-65ce02d460d8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:06.047718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc827-cb5d-488d-8285-65ce02d460d8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:06.047850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc827-cb5d-488d-8285-65ce02d460d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:06.129835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-658bc827-cb5d-488d-8285-65ce02d460d8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:06.131232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-658bc827-cb5d-488d-8285-65ce02d460d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:06.176682 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:06.241100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 561/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:33:06.400800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2844993-21ed-4954-9c6d-5f9df2354d74 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:06.402361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b2844993-21ed-4954-9c6d-5f9df2354d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:06.572676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2844993-21ed-4954-9c6d-5f9df2354d74 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:06.631851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:06.696570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:06.697417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:06.697744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:06.698064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:06.699715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:06.804790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:06.806596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6e398-ecd5-42b9-b0d1-93d62223bd3b service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:06.826617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 553/1114] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:08.280720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694481d-417d-4b9d-877c-9dba97600949 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:08.280720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694481d-417d-4b9d-877c-9dba97600949 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:08.280720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694481d-417d-4b9d-877c-9dba97600949 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:08.280720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 562/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:08.283043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9fe8a-45e3-414a-a582-cbb9bf39a4b6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:08.283473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9fe8a-45e3-414a-a582-cbb9bf39a4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:08.283591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9fe8a-45e3-414a-a582-cbb9bf39a4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:08.365650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-96d9fe8a-45e3-414a-a582-cbb9bf39a4b6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:08.366877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-96d9fe8a-45e3-414a-a582-cbb9bf39a4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:08.369017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:08.488027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 554/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:08.643523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694481d-417d-4b9d-877c-9dba97600949 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:08.645100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7694481d-417d-4b9d-877c-9dba97600949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:08.802305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694481d-417d-4b9d-877c-9dba97600949 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:08.831470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:08.924641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:08.925301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:08.925570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:08.925840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:08.927467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:09.021564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:09.023148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d6d59-91bc-471a-ac7a-1f7f27948e92 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:09.042174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 563/1117] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:11.212249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef479fdb-1d0e-4de5-97c8-2aefc6e88549 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:11.212819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef479fdb-1d0e-4de5-97c8-2aefc6e88549 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:11.212819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef479fdb-1d0e-4de5-97c8-2aefc6e88549 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:11.214130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 555/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:11.222230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34c54c-9fb1-47cc-bcf1-da2618cc0895 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:11.222776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34c54c-9fb1-47cc-bcf1-da2618cc0895 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:11.223070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34c54c-9fb1-47cc-bcf1-da2618cc0895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:11.283640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:11.316856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ac34c54c-9fb1-47cc-bcf1-da2618cc0895 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:11.318096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ac34c54c-9fb1-47cc-bcf1-da2618cc0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:11.433094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 564/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:33:11.725226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef479fdb-1d0e-4de5-97c8-2aefc6e88549 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:11.728163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ef479fdb-1d0e-4de5-97c8-2aefc6e88549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:11.854684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef479fdb-1d0e-4de5-97c8-2aefc6e88549 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:11.892893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:11.946334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:11.946966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:11.947232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:11.947495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:11.949218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:12.065957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:12.067310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb5584-92a1-44f0-bc3c-951f2b289848 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:12.088066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 556/1120] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:14.600033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d762529-5773-4bbe-a548-922140d314c6 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:14.600600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d762529-5773-4bbe-a548-922140d314c6 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:14.600600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d762529-5773-4bbe-a548-922140d314c6 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:14.602149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 565/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:14.609414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8763b8d-7272-4d3f-a801-67496d1a7550 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:14.610048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8763b8d-7272-4d3f-a801-67496d1a7550 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:14.610048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8763b8d-7272-4d3f-a801-67496d1a7550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:14.683722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f8763b8d-7272-4d3f-a801-67496d1a7550 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:14.685096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f8763b8d-7272-4d3f-a801-67496d1a7550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:14.699683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:14.791778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 557/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:15.324681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d762529-5773-4bbe-a548-922140d314c6 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:15.327136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9d762529-5773-4bbe-a548-922140d314c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:15.488076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d762529-5773-4bbe-a548-922140d314c6 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:15.503307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:15.607847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:15.609308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:15.609750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:15.610183 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:15.613058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:15.776269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:15.779359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c048c-f1c9-4c5d-9a20-00dafdf2b7de service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:15.806412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 566/1123] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:18.515970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a6a0-35a8-4d4e-97ba-f29bdfa818e4 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:18.516587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a6a0-35a8-4d4e-97ba-f29bdfa818e4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:18.516587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a6a0-35a8-4d4e-97ba-f29bdfa818e4 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:18.517914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 558/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:33:18.524342 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca79c-d068-4660-8c27-da75721718ad None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:18.524886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca79c-d068-4660-8c27-da75721718ad None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:18.524982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca79c-d068-4660-8c27-da75721718ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:18.575835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:18.596193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d76ca79c-d068-4660-8c27-da75721718ad None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:18.597474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d76ca79c-d068-4660-8c27-da75721718ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:18.702953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 567/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:18.996726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a6a0-35a8-4d4e-97ba-f29bdfa818e4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:18.998283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0c67a6a0-35a8-4d4e-97ba-f29bdfa818e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:19.109254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a6a0-35a8-4d4e-97ba-f29bdfa818e4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:19.151126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:19.188702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:19.189260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:19.189541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:19.189813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:19.191896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:19.322132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:19.323585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b56637-13ea-467c-9c25-9d2c812e68d0 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:19.343817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 559/1126] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:21.835070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c3346-a9a1-4906-bc67-6ee68be70904 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:21.835745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c3346-a9a1-4906-bc67-6ee68be70904 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:21.835745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c3346-a9a1-4906-bc67-6ee68be70904 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:21.837137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 568/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:21.847187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243cc0-da45-46ea-bdff-b38b62a47f7c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:21.847804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243cc0-da45-46ea-bdff-b38b62a47f7c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:21.848138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243cc0-da45-46ea-bdff-b38b62a47f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:21.962130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2b243cc0-da45-46ea-bdff-b38b62a47f7c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:21.964344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2b243cc0-da45-46ea-bdff-b38b62a47f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:22.003012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:22.119907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 560/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:22.417750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c3346-a9a1-4906-bc67-6ee68be70904 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:22.419461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ee8c3346-a9a1-4906-bc67-6ee68be70904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:22.603561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c3346-a9a1-4906-bc67-6ee68be70904 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:22.642908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:22.705439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:22.706113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:22.706396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:22.706672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:22.708313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:22.841588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:22.843480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d812b1-4bfc-4877-a8a2-025024d3bd0b service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:22.864693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 569/1129] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:25.329029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b9757-2aa4-4e09-9c2b-35e62a2dec7d None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:25.329567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b9757-2aa4-4e09-9c2b-35e62a2dec7d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:25.329567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b9757-2aa4-4e09-9c2b-35e62a2dec7d None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:25.330954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 561/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:25.336641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd8920-8b23-4507-b200-04b9754431e8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:25.337135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd8920-8b23-4507-b200-04b9754431e8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:25.337250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd8920-8b23-4507-b200-04b9754431e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:25.415584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:25.424858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-66cd8920-8b23-4507-b200-04b9754431e8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:25.424858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-66cd8920-8b23-4507-b200-04b9754431e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:25.533973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 570/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:25.818987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b9757-2aa4-4e09-9c2b-35e62a2dec7d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:25.820669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-328b9757-2aa4-4e09-9c2b-35e62a2dec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:25.935478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b9757-2aa4-4e09-9c2b-35e62a2dec7d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:25.988396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:26.020862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:26.021503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:26.021777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:26.022046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:26.024581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:26.141346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:26.146932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb003c4-5d6c-4555-8984-8c6eb24fc2f1 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:26.165729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 562/1132] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:28.572168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c0ea5-5bc4-40fa-969a-e181565e0db8 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:28.572828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c0ea5-5bc4-40fa-969a-e181565e0db8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:28.572828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c0ea5-5bc4-40fa-969a-e181565e0db8 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:28.574211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 571/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:28.581293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774ea0-c190-4627-b19d-04e806dc22f3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:28.581841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774ea0-c190-4627-b19d-04e806dc22f3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:28.581985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774ea0-c190-4627-b19d-04e806dc22f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:28.666880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-34774ea0-c190-4627-b19d-04e806dc22f3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:28.668038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-34774ea0-c190-4627-b19d-04e806dc22f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:28.703450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:28.788248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 563/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:29.084003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c0ea5-5bc4-40fa-969a-e181565e0db8 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:29.085585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-747c0ea5-5bc4-40fa-969a-e181565e0db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:29.207603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c0ea5-5bc4-40fa-969a-e181565e0db8 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:29.242317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:29.295144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:29.295797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:29.296074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:29.296343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:29.298066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:29.445418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:29.446878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9acf1-dabe-417f-b2ab-ba9f1fe3f237 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:29.468338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 572/1135] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:32.634782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7437-7844-45dd-bdfd-6506e5bef41c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:32.635802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7437-7844-45dd-bdfd-6506e5bef41c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:32.635802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7437-7844-45dd-bdfd-6506e5bef41c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:32.637456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 564/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:32.643877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae01f19-e6fd-49c7-b30c-2d627ddaf068 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:32.644536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae01f19-e6fd-49c7-b30c-2d627ddaf068 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:32.644536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae01f19-e6fd-49c7-b30c-2d627ddaf068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:32.736318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5ae01f19-e6fd-49c7-b30c-2d627ddaf068 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:32.738445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5ae01f19-e6fd-49c7-b30c-2d627ddaf068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:32.754485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:32.904490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 573/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:33.194989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7437-7844-45dd-bdfd-6506e5bef41c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:33.196621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6f2b7437-7844-45dd-bdfd-6506e5bef41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:33.362437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7437-7844-45dd-bdfd-6506e5bef41c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:33.431796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:33.454776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:33.455314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:33.455592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:33.455889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:33.457648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:33.590421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:33.593246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9477c0-c506-409f-af70-48fcacfb03ca service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:33.625341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 565/1138] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:33:36.871737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1659-6e6c-4c7c-8035-d45bafff80a5 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:36.872562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1659-6e6c-4c7c-8035-d45bafff80a5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:36.872804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1659-6e6c-4c7c-8035-d45bafff80a5 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:36.875382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 574/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:36.882174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca386c6-71a5-45bb-8f0c-4e8c5ab62873 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:36.882741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca386c6-71a5-45bb-8f0c-4e8c5ab62873 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:36.882741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca386c6-71a5-45bb-8f0c-4e8c5ab62873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:36.959407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7ca386c6-71a5-45bb-8f0c-4e8c5ab62873 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:36.960637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7ca386c6-71a5-45bb-8f0c-4e8c5ab62873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:37.005352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:37.078167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 566/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:33:37.379468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1659-6e6c-4c7c-8035-d45bafff80a5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:37.382312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-68be1659-6e6c-4c7c-8035-d45bafff80a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:37.546768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1659-6e6c-4c7c-8035-d45bafff80a5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:37.602256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:37.647910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:37.648563 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:37.648869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:37.649188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:37.650916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:37.790186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:37.791625 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4b57b-c9ea-4314-9842-77e283594cc9 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:37.812972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 575/1141] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:40.723481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f54520-6b17-463f-a6cf-c4452dd14a71 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:40.724279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f54520-6b17-463f-a6cf-c4452dd14a71 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:40.724279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f54520-6b17-463f-a6cf-c4452dd14a71 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:40.725386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 567/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:33:40.734071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7353dc-3213-4ae0-9e60-273bac4de63c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:40.734624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7353dc-3213-4ae0-9e60-273bac4de63c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:40.735072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7353dc-3213-4ae0-9e60-273bac4de63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:40.834254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2f7353dc-3213-4ae0-9e60-273bac4de63c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:40.835545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2f7353dc-3213-4ae0-9e60-273bac4de63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:40.856544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:40.949876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 576/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:41.250650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f54520-6b17-463f-a6cf-c4452dd14a71 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:41.252447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d4f54520-6b17-463f-a6cf-c4452dd14a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:41.385108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f54520-6b17-463f-a6cf-c4452dd14a71 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:41.464643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:41.473237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:41.473897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:41.474215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:41.474539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:41.476357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:41.633068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:41.634668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76381b68-82aa-4fb4-8d33-1fef40531f29 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:41.660374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 568/1144] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:33:43.960232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c07b0-23c3-4189-9e55-0c7b0f0c8ca2 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:43.960815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c07b0-23c3-4189-9e55-0c7b0f0c8ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:43.960815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c07b0-23c3-4189-9e55-0c7b0f0c8ca2 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:43.962265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 577/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:33:43.969939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5c02b-d680-47a7-b87b-1c19b8e32067 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:43.970471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5c02b-d680-47a7-b87b-1c19b8e32067 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:43.970897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5c02b-d680-47a7-b87b-1c19b8e32067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:44.096400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cec5c02b-d680-47a7-b87b-1c19b8e32067 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:33:44.098564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cec5c02b-d680-47a7-b87b-1c19b8e32067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:44.182583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:44.236924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 569/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:33:44.538425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c07b0-23c3-4189-9e55-0c7b0f0c8ca2 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:33:44.540036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e23c07b0-23c3-4189-9e55-0c7b0f0c8ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:44.668099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c07b0-23c3-4189-9e55-0c7b0f0c8ca2 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:33:44.731121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:44.763988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:33:44.764607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:33:44.764888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:33:44.765193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:33:44.766965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:33:44.906935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:33:44.907934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1302b2-e98b-49b8-bf25-448db02b442b service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:33:44.930190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 578/1147] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:33:54.235296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=22 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:33:54.927664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=66 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:06.848417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db4c2-a053-427c-9464-d4aa932d411a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:06.849107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db4c2-a053-427c-9464-d4aa932d411a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:06.849107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db4c2-a053-427c-9464-d4aa932d411a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:06.850711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 570/1148] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:34:06.857351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee1f66-1a5c-4765-bd1a-022d3addd609 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:06.858122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee1f66-1a5c-4765-bd1a-022d3addd609 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:06.858122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee1f66-1a5c-4765-bd1a-022d3addd609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:06.942541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-55ee1f66-1a5c-4765-bd1a-022d3addd609 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:06.944140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-55ee1f66-1a5c-4765-bd1a-022d3addd609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:07.073452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 579/1149] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:07.378819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0dce1-e8e0-42ac-a11b-fc7c48a13f70 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:07.379103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0dce1-e8e0-42ac-a11b-fc7c48a13f70 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:07.379382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0dce1-e8e0-42ac-a11b-fc7c48a13f70 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:07.380796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 571/1150] 10.222.0.30 () {54 vars in 962 bytes} [Wed Nov 2 14:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:34:07.387567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a20f5-f7dc-4527-9f34-d0fb6501c96f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:07.388005 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a20f5-f7dc-4527-9f34-d0fb6501c96f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:07.388540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a20f5-f7dc-4527-9f34-d0fb6501c96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:07.484656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-136a20f5-f7dc-4527-9f34-d0fb6501c96f None None] MFA Rules not processed for user `287fbe09913b454c9fcd58febbca6b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:07.485926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-136a20f5-f7dc-4527-9f34-d0fb6501c96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:07.509306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:07.575889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 580/1151] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Nov 2 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:07.584224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0dce1-e8e0-42ac-a11b-fc7c48a13f70 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:07.585733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-13e0dce1-e8e0-42ac-a11b-fc7c48a13f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:07.700239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0dce1-e8e0-42ac-a11b-fc7c48a13f70 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:07.780357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:07.780966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:07.781272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:07.781541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:07.783160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:07.840850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:07.906432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:07.907878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a2660-2b1b-4ce4-a0e7-8c18cb184ff1 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:07.928348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 572/1152] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:34:10.297328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338060-9ce0-4e8d-aa72-d0d9cc1818af None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:10.297912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338060-9ce0-4e8d-aa72-d0d9cc1818af None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:10.297912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338060-9ce0-4e8d-aa72-d0d9cc1818af None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:10.299144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 581/1153] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:34:10.305387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe8c5c-700f-4da5-a5a5-69168c9e14d5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:10.305912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe8c5c-700f-4da5-a5a5-69168c9e14d5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:10.306063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe8c5c-700f-4da5-a5a5-69168c9e14d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:10.383170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-70fe8c5c-700f-4da5-a5a5-69168c9e14d5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:10.384434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-70fe8c5c-700f-4da5-a5a5-69168c9e14d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:10.501640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 573/1154] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:10.783118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338060-9ce0-4e8d-aa72-d0d9cc1818af None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:10.784671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bb338060-9ce0-4e8d-aa72-d0d9cc1818af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:10.945719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338060-9ce0-4e8d-aa72-d0d9cc1818af None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:11.008533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:11.060122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:11.060650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:11.060926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:11.061224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:11.061871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:11.062987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:34:11.062987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:34:11.062987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:34:11.064180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c295bc1-9082-4cc7-92d4-5da5986aed07 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:11.086681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 582/1155] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 14:34:10 2022] GET /identity/v3/projects => generated 2553 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:34:13.378044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6bbb-71e3-4b69-a0c0-10c8d5d8b6bd None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:13.378676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6bbb-71e3-4b69-a0c0-10c8d5d8b6bd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:13.378676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6bbb-71e3-4b69-a0c0-10c8d5d8b6bd None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:13.379945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 574/1156] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:34:13.386073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759ca45-fdc0-40fa-a1b4-9f935183119f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:13.386589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759ca45-fdc0-40fa-a1b4-9f935183119f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:13.386589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759ca45-fdc0-40fa-a1b4-9f935183119f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:13.501486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e759ca45-fdc0-40fa-a1b4-9f935183119f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:13.503562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e759ca45-fdc0-40fa-a1b4-9f935183119f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:13.643053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 583/1157] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:13.949882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6bbb-71e3-4b69-a0c0-10c8d5d8b6bd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:13.951810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-357a6bbb-71e3-4b69-a0c0-10c8d5d8b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:13.962320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:14.112413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6bbb-71e3-4b69-a0c0-10c8d5d8b6bd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:14.205612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:14.251507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:14.252152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:14.252429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:14.252704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:14.254540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:14.382124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:14.383834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a537d8-42f5-4184-b3a1-a026bac69c2e service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:14.407397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 575/1158] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:34:16.882500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23d12a-7722-4f66-bfb8-96ec5caa0d87 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:16.883116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23d12a-7722-4f66-bfb8-96ec5caa0d87 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:16.883116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23d12a-7722-4f66-bfb8-96ec5caa0d87 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:16.884399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 584/1159] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:34:16.891455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e52c2-e3ed-4508-8166-21527b3aa8e9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:16.891969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e52c2-e3ed-4508-8166-21527b3aa8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:16.892075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e52c2-e3ed-4508-8166-21527b3aa8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:16.967455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c79e52c2-e3ed-4508-8166-21527b3aa8e9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:16.968733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c79e52c2-e3ed-4508-8166-21527b3aa8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:17.026048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:17.085720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 576/1160] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:17.389418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23d12a-7722-4f66-bfb8-96ec5caa0d87 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:17.390981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1e23d12a-7722-4f66-bfb8-96ec5caa0d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:17.416574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:17.503850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23d12a-7722-4f66-bfb8-96ec5caa0d87 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:17.582977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:17.583582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:17.583860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:17.584128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:17.585824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:17.668772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:17.713326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:17.714886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8adf8-4211-45b4-83d2-730e5fa77d5f service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:17.739449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 585/1161] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:34:20.055223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1f09-03fc-4ea0-bba3-154d32516a5c None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:20.055789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1f09-03fc-4ea0-bba3-154d32516a5c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:20.055789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1f09-03fc-4ea0-bba3-154d32516a5c None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:20.057197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 577/1162] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:34:20.063002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf9430-f160-4c38-b615-000e1e8ffb08 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:20.063525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf9430-f160-4c38-b615-000e1e8ffb08 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:20.063632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf9430-f160-4c38-b615-000e1e8ffb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:20.134316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-52cf9430-f160-4c38-b615-000e1e8ffb08 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:20.135539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-52cf9430-f160-4c38-b615-000e1e8ffb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:20.245576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 586/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:20.536317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e1f09-03fc-4ea0-bba3-154d32516a5c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:20.537875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c97e1f09-03fc-4ea0-bba3-154d32516a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:20.672750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e1f09-03fc-4ea0-bba3-154d32516a5c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:20.752066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:20.753502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:20.754179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:20.754442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:20.754727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:20.756374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:20.895760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:20.897233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1384e9a-b40a-4e93-b074-d3d2f31c6f60 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:20.918277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 578/1164] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:34:23.434288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85920e-540f-4ef0-bf6e-79d53ecb9ca9 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:23.435071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85920e-540f-4ef0-bf6e-79d53ecb9ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:23.435071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85920e-540f-4ef0-bf6e-79d53ecb9ca9 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:23.436276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 587/1165] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:34:23.442202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39121b3-fb55-4310-b29e-384a892ecd90 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:23.443894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39121b3-fb55-4310-b29e-384a892ecd90 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:23.443894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39121b3-fb55-4310-b29e-384a892ecd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:23.516051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a39121b3-fb55-4310-b29e-384a892ecd90 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:23.517326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a39121b3-fb55-4310-b29e-384a892ecd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:23.575649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:23.622953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 579/1166] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:23.926165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85920e-540f-4ef0-bf6e-79d53ecb9ca9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:23.927750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2d85920e-540f-4ef0-bf6e-79d53ecb9ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:23.970061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:24.047683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85920e-540f-4ef0-bf6e-79d53ecb9ca9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:24.125934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:24.126614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:24.126889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:24.127158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:24.128807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:24.210698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:24.240719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:24.242498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c7048-ceba-45bc-8504-e98cf881ae6d service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:24.267398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 588/1167] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:34:26.877495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5c8f1-bff7-4007-b464-01be1b5008d6 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:26.878190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5c8f1-bff7-4007-b464-01be1b5008d6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:26.878190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5c8f1-bff7-4007-b464-01be1b5008d6 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:26.879365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 580/1168] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:34:26.885123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9dd9-7bb4-4a9f-9daa-9cd2eb58c8a1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:26.885580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9dd9-7bb4-4a9f-9daa-9cd2eb58c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:26.885660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9dd9-7bb4-4a9f-9daa-9cd2eb58c8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:26.959020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-baec9dd9-7bb4-4a9f-9daa-9cd2eb58c8a1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:34:26.960257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-baec9dd9-7bb4-4a9f-9daa-9cd2eb58c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:27.065693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 589/1169] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 14:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:34:27.367995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5c8f1-bff7-4007-b464-01be1b5008d6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:34:27.370327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-20d5c8f1-bff7-4007-b464-01be1b5008d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:27.488723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5c8f1-bff7-4007-b464-01be1b5008d6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:34:27.571279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:34:27.571891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:34:27.572168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:34:27.572438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:34:27.574083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:34:27.583210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:27.696103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:34:27.697601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4441f20-4009-4621-a9cb-35f3ba6196a0 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:34:27.717290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 581/1170] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:34:37.064072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=82 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:34:37.714581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=50 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:35:51.558272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:51.558899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:51.558899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:51.650412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:35:51.651542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:51.773621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 590/1171] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:35:51.781635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:51.782124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:51.782229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:51.804596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:35:51.805837 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:51.814435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 582/1172] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:35:51.818995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:51.829154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020320c-dd71-473e-a65b-f53946f8d5aa None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:51.833861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:51.834473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:51.835366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:51.836077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:51.836326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:35:51.836626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:51.837680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:35:51.837680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:35:51.837680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:35:51.838575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:51.851198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 591/1173] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:35:51.864318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:51.875055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91272d0-4ee6-4563-bba3-e378adf2d003 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:51.879838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:35:51.885881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:35:51.886503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:51.988795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 583/1174] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:35:51 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:35:51.993140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:52.003070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea699e-e5f3-40f4-87fe-3206dd3cc0ac admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:52.052943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:35:52.059900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:52.060534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.062082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.062082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.062082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:52.062706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:35:52.062706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:35:52.062706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:35:52.063933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:52.136839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 592/1175] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:35:51 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:35:52.141146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:52.152321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582bfc-0277-43d4-8ec4-2130494fa6ab admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:52.156854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:52.157622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.157835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.158142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.158732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:52.160330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:52.173711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 584/1176] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:35:52.178028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:52.189075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134acce5-b929-40f2-98c9-7785818c630d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:52.193971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5bf90-b709-4fbf-981d-da82b33aa925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:52.195094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5bf90-b709-4fbf-981d-da82b33aa925 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.195369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5bf90-b709-4fbf-981d-da82b33aa925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.195692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5bf90-b709-4fbf-981d-da82b33aa925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.196313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5bf90-b709-4fbf-981d-da82b33aa925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:52.197941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5bf90-b709-4fbf-981d-da82b33aa925 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:52.212669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 593/1177] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:35:52.217079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:52.227089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff194d-7be7-49c3-bb54-e421342f321d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:52.231952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80926-8539-4dfb-bd09-8aad673e33be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:52.232604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80926-8539-4dfb-bd09-8aad673e33be admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.232880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80926-8539-4dfb-bd09-8aad673e33be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.233184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80926-8539-4dfb-bd09-8aad673e33be admin admin] PATH_INFO: `/v3/projects/1219c664973246c78adb13af22d864f0/users/04d6f3056c54422886c2656570c744be/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.248321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80926-8539-4dfb-bd09-8aad673e33be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219c664973246c78adb13af22d864f0, user_id=04d6f3056c54422886c2656570c744be, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:52.251026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80926-8539-4dfb-bd09-8aad673e33be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:52.330711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 585/1178] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:35:52 2022] PUT /identity/v3/projects/1219c664973246c78adb13af22d864f0/users/04d6f3056c54422886c2656570c744be/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:35:52.343367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0156-8451-4410-b3af-12607767c249 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.343708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0156-8451-4410-b3af-12607767c249 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.344032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0156-8451-4410-b3af-12607767c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.374775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1a4d0156-8451-4410-b3af-12607767c249 None None] MFA Rules not processed for user `04d6f3056c54422886c2656570c744be`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:35:52.375984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1a4d0156-8451-4410-b3af-12607767c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:52.440332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 594/1179] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:35:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:35:52.454336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d5731-9032-45d9-892b-6fed0f97dc2a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.454620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d5731-9032-45d9-892b-6fed0f97dc2a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.454889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d5731-9032-45d9-892b-6fed0f97dc2a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.456264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 586/1180] 10.222.0.30 () {54 vars in 963 bytes} [Wed Nov 2 14:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:35:52.461899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3874-6488-4451-861c-08d4b69d90b1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.462185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3874-6488-4451-861c-08d4b69d90b1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.462462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3874-6488-4451-861c-08d4b69d90b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.531565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-8edb3874-6488-4451-861c-08d4b69d90b1 None None] MFA Rules not processed for user `20e8f2a80ea142b7acb78ea661b0e47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:35:52.532794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8edb3874-6488-4451-861c-08d4b69d90b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:52.616773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 595/1181] 10.222.0.30 () {58 vars in 1060 bytes} [Wed Nov 2 14:35:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:35:52.625034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d5731-9032-45d9-892b-6fed0f97dc2a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:52.636888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d5731-9032-45d9-892b-6fed0f97dc2a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:52.640485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:52.641143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.641424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.641693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.650860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:52.652281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:52.661532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 587/1182] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:35:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:35:52.708207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c95cf-95c4-44f3-adc9-c1fcdc3258ed None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.708487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c95cf-95c4-44f3-adc9-c1fcdc3258ed None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.708756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c95cf-95c4-44f3-adc9-c1fcdc3258ed None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.710617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 596/1183] 10.222.0.30 () {52 vars in 830 bytes} [Wed Nov 2 14:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:35:52.714308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:52.781393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f5672-ac19-423a-b71b-faaf9febc849 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:52.786018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:52.786645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:52.786922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:52.787193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] PATH_INFO: `/v3/users/04d6f3056c54422886c2656570c744be` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:52.788898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d6f3056c54422886c2656570c744be)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:52.789944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:35:52.789944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:35:52.789944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:35:52.791218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:53.002856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d6f3056c54422886c2656570c744be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:53.027998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d6f3056c54422886c2656570c744be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:53.106768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d6f3056c54422886c2656570c744be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:53.107392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d6f3056c54422886c2656570c744be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:53.108126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d6f3056c54422886c2656570c744be'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:53.119673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:35:53.129353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 588/1184] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:35:52 2022] DELETE /identity/v3/users/04d6f3056c54422886c2656570c744be => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:35:53.134418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:53.134685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:53.134951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:53.154277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:35:53.155465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:53.164937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:35:53.215261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 597/1185] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:35:53.224785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:53.227126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:53.308571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5f57a-c937-4b46-b2bd-814931ae4779 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:53.312627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e339d5-3efb-4c05-b35d-42df60143503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:53.313294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e339d5-3efb-4c05-b35d-42df60143503 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:53.313575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e339d5-3efb-4c05-b35d-42df60143503 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:53.313841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e339d5-3efb-4c05-b35d-42df60143503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:53.323890 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e339d5-3efb-4c05-b35d-42df60143503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:53.325368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e339d5-3efb-4c05-b35d-42df60143503 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:53.334909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 589/1186] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:35:53.870233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:35:53.871680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:35:53.883572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafae48-f4ff-4ace-9e51-e3a3e56387ac None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:35:53.892052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:35:53.892693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:35:53.892970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:35:53.893327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] PATH_INFO: `/v3/projects/1219c664973246c78adb13af22d864f0` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:35:53.897040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1219c664973246c78adb13af22d864f0)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:35:53.897040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:35:53.897040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:35:53.897040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:35:53.897040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:35:54.075400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1219c664973246c78adb13af22d864f0', 'request_id': 'req-74e41c32-6d42-4d15-995d-40de080a8351'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:54.091463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1219c664973246c78adb13af22d864f0', 'request_id': 'req-74e41c32-6d42-4d15-995d-40de080a8351'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:54.175900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1219c664973246c78adb13af22d864f0', 'request_id': 'req-74e41c32-6d42-4d15-995d-40de080a8351'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:54.187633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] The token cache is being invalidate because project 1219c664973246c78adb13af22d864f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:35:54.187882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:54.189134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-74e41c32-6d42-4d15-995d-40de080a8351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:35:54.190110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 598/1187] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:35:53 2022] DELETE /identity/v3/projects/1219c664973246c78adb13af22d864f0 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:36:03.330771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=38 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:36:04.184587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:37:22.655791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa601e-7987-4505-8568-fff6548551c1 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:22.656419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa601e-7987-4505-8568-fff6548551c1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:22.656419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa601e-7987-4505-8568-fff6548551c1 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:22.657749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 590/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:22.663101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9241a5-a3dc-40e7-9355-b61f85d71933 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:22.663530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9241a5-a3dc-40e7-9355-b61f85d71933 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:22.663660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9241a5-a3dc-40e7-9355-b61f85d71933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:22.683690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ad9241a5-a3dc-40e7-9355-b61f85d71933 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:22.684909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ad9241a5-a3dc-40e7-9355-b61f85d71933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:22.743946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 599/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:22.900590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa601e-7987-4505-8568-fff6548551c1 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:22.902414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-18fa601e-7987-4505-8568-fff6548551c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:23.007602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa601e-7987-4505-8568-fff6548551c1 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:23.013269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57690d-ae58-4b53-953a-d0b56b504b26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:23.013883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57690d-ae58-4b53-953a-d0b56b504b26 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:23.014163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57690d-ae58-4b53-953a-d0b56b504b26 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:23.014433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57690d-ae58-4b53-953a-d0b56b504b26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:23.023331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57690d-ae58-4b53-953a-d0b56b504b26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:23.024922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57690d-ae58-4b53-953a-d0b56b504b26 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:23.034185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 591/1190] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:24.106595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742541d4-3373-476b-a275-dfef536c8c1f None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:24.107146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742541d4-3373-476b-a275-dfef536c8c1f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:24.107146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742541d4-3373-476b-a275-dfef536c8c1f None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:24.108570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 600/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:24.113526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef373-a539-4a19-a139-e905e6f9f55d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:24.113959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef373-a539-4a19-a139-e905e6f9f55d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:24.114172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef373-a539-4a19-a139-e905e6f9f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:24.134133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6a9ef373-a539-4a19-a139-e905e6f9f55d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:24.135335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6a9ef373-a539-4a19-a139-e905e6f9f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:24.143181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 592/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:24.295417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742541d4-3373-476b-a275-dfef536c8c1f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:24.309724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742541d4-3373-476b-a275-dfef536c8c1f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:24.315095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f932a-1c9e-4163-96b2-5b9be013338e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:24.315689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f932a-1c9e-4163-96b2-5b9be013338e service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:24.315970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f932a-1c9e-4163-96b2-5b9be013338e service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:24.316238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f932a-1c9e-4163-96b2-5b9be013338e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:24.324881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f932a-1c9e-4163-96b2-5b9be013338e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:24.326505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f932a-1c9e-4163-96b2-5b9be013338e service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:24.336566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 601/1193] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:24.404280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b5407-0944-453a-83a6-dd0114024747 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:24.404568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b5407-0944-453a-83a6-dd0114024747 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:24.404841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b5407-0944-453a-83a6-dd0114024747 None None] PATH_INFO: `/v3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:24.406225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 593/1194] 10.222.0.64 () {54 vars in 925 bytes} [Wed Nov 2 14:37:24 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:37:24.413539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fe427-55af-4a02-a55c-5315f206bc28 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:24.413924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fe427-55af-4a02-a55c-5315f206bc28 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:24.414293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fe427-55af-4a02-a55c-5315f206bc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:24.471087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-491fe427-55af-4a02-a55c-5315f206bc28 None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:24.472329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-491fe427-55af-4a02-a55c-5315f206bc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:24.548295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 602/1195] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Nov 2 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:37:33.482245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa04bb-6950-4b17-8524-dadea2208c17 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:33.483012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa04bb-6950-4b17-8524-dadea2208c17 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:33.483198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa04bb-6950-4b17-8524-dadea2208c17 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:33.484612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 594/1196] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 14:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:33.489571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02d0b-5829-4c54-9b8d-be6181637e2a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:33.490033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02d0b-5829-4c54-9b8d-be6181637e2a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:33.490198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02d0b-5829-4c54-9b8d-be6181637e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:33.509472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-24f02d0b-5829-4c54-9b8d-be6181637e2a None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:33.510665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-24f02d0b-5829-4c54-9b8d-be6181637e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:33.519071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 603/1197] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 14:37:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:37:33.527174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa04bb-6950-4b17-8524-dadea2208c17 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:33.541127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa04bb-6950-4b17-8524-dadea2208c17 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:33.546290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88e9f-fa35-40bd-a432-e452d588f720 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:33.546875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88e9f-fa35-40bd-a432-e452d588f720 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:33.547144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88e9f-fa35-40bd-a432-e452d588f720 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:33.547407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88e9f-fa35-40bd-a432-e452d588f720 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:33.555839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88e9f-fa35-40bd-a432-e452d588f720 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:33.557455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88e9f-fa35-40bd-a432-e452d588f720 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:33.566808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 595/1198] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:35.817521 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d942-2bcd-461b-862e-e9d538a0a4c7 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:35.818045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d942-2bcd-461b-862e-e9d538a0a4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:35.818045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d942-2bcd-461b-862e-e9d538a0a4c7 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:35.819999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 604/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:35.825256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf0de5-5a11-4323-b58a-a056974cee7f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:35.825680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf0de5-5a11-4323-b58a-a056974cee7f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:35.825796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf0de5-5a11-4323-b58a-a056974cee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:35.848177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-83bf0de5-5a11-4323-b58a-a056974cee7f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:35.849501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-83bf0de5-5a11-4323-b58a-a056974cee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:35.857804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 596/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:36.018014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d942-2bcd-461b-862e-e9d538a0a4c7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:36.032440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d942-2bcd-461b-862e-e9d538a0a4c7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:36.038623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea89e4-47a7-4eba-bc76-43023006c860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:36.039252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea89e4-47a7-4eba-bc76-43023006c860 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:36.039533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea89e4-47a7-4eba-bc76-43023006c860 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:36.039802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea89e4-47a7-4eba-bc76-43023006c860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:36.048060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea89e4-47a7-4eba-bc76-43023006c860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:36.049679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea89e4-47a7-4eba-bc76-43023006c860 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:36.059993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 605/1201] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:37.153643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f92af6-a262-4d13-ae43-351580a56a49 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:37.154211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f92af6-a262-4d13-ae43-351580a56a49 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:37.154211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f92af6-a262-4d13-ae43-351580a56a49 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:37.155537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 597/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:37.160598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37fc0d-1bab-4aad-9137-c3d93c10a903 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:37.161091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37fc0d-1bab-4aad-9137-c3d93c10a903 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:37.161214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37fc0d-1bab-4aad-9137-c3d93c10a903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:37.181238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-7d37fc0d-1bab-4aad-9137-c3d93c10a903 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:37.182485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7d37fc0d-1bab-4aad-9137-c3d93c10a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:37.190377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 606/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 14:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:37.344442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f92af6-a262-4d13-ae43-351580a56a49 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:37.360380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f92af6-a262-4d13-ae43-351580a56a49 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:37.365912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a96ef-756d-48c8-93a9-1d1e6c985fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:37.366566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a96ef-756d-48c8-93a9-1d1e6c985fec service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:37.366839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a96ef-756d-48c8-93a9-1d1e6c985fec service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:37.367106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a96ef-756d-48c8-93a9-1d1e6c985fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:37.375199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a96ef-756d-48c8-93a9-1d1e6c985fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:37.376786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a96ef-756d-48c8-93a9-1d1e6c985fec service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:37.386365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 598/1204] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:41.347472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ef56d-e363-4470-b3f4-c76e08e4293c None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:41.348000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ef56d-e363-4470-b3f4-c76e08e4293c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:41.348000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ef56d-e363-4470-b3f4-c76e08e4293c None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:41.349305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 607/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:41.355367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b43b88-9883-436b-9ccd-b812286e2600 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:41.355782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b43b88-9883-436b-9ccd-b812286e2600 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:41.355903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b43b88-9883-436b-9ccd-b812286e2600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:41.375717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-02b43b88-9883-436b-9ccd-b812286e2600 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:41.376898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-02b43b88-9883-436b-9ccd-b812286e2600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:41.385223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 599/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:41.677885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ef56d-e363-4470-b3f4-c76e08e4293c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:41.679904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-977ef56d-e363-4470-b3f4-c76e08e4293c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:41.710398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:37:41.741814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ef56d-e363-4470-b3f4-c76e08e4293c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:41.745709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425e028-726e-4b09-8953-e017c1bea9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:41.746319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e028-726e-4b09-8953-e017c1bea9ef service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:41.746590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e028-726e-4b09-8953-e017c1bea9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:41.746857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e028-726e-4b09-8953-e017c1bea9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:41.756854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e028-726e-4b09-8953-e017c1bea9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:41.758423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e028-726e-4b09-8953-e017c1bea9ef service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:41.768498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 608/1207] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:37:43.955815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c9fe8-7b28-40cb-ae05-b3bd91f7e8a8 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:43.956391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c9fe8-7b28-40cb-ae05-b3bd91f7e8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:43.956391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c9fe8-7b28-40cb-ae05-b3bd91f7e8a8 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:43.957724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 600/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:43.963963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf723db-f5b0-4081-bdfa-c5b01550efbf None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:43.964429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf723db-f5b0-4081-bdfa-c5b01550efbf None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:43.964570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf723db-f5b0-4081-bdfa-c5b01550efbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:43.984714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-bbf723db-f5b0-4081-bdfa-c5b01550efbf None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:43.985928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bbf723db-f5b0-4081-bdfa-c5b01550efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:43.993898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 609/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:44.281802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c9fe8-7b28-40cb-ae05-b3bd91f7e8a8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:44.283533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-235c9fe8-7b28-40cb-ae05-b3bd91f7e8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:44.291777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=68 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:37:44.297523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c9fe8-7b28-40cb-ae05-b3bd91f7e8a8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:44.301175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe01751-3694-468b-9949-46b85569e223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:44.301778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe01751-3694-468b-9949-46b85569e223 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:44.302079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe01751-3694-468b-9949-46b85569e223 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:44.302350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe01751-3694-468b-9949-46b85569e223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:44.311733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe01751-3694-468b-9949-46b85569e223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:44.313177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe01751-3694-468b-9949-46b85569e223 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:44.322618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 601/1210] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:48.037993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174ea2-e210-4080-80ca-435b0c84687a None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:48.038552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174ea2-e210-4080-80ca-435b0c84687a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:48.038552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174ea2-e210-4080-80ca-435b0c84687a None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:48.040384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 610/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:48.045735 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9af8bc-c443-4627-8252-5aceaaea9f1a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:48.046152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9af8bc-c443-4627-8252-5aceaaea9f1a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:48.046274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9af8bc-c443-4627-8252-5aceaaea9f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:48.066502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ef9af8bc-c443-4627-8252-5aceaaea9f1a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:48.067710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ef9af8bc-c443-4627-8252-5aceaaea9f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:48.075825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 602/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:37:48.385115 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174ea2-e210-4080-80ca-435b0c84687a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:48.386821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-35174ea2-e210-4080-80ca-435b0c84687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:48.445249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174ea2-e210-4080-80ca-435b0c84687a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:48.449420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83216f58-419d-4def-8f8b-bc7f07f1d3f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:48.450063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83216f58-419d-4def-8f8b-bc7f07f1d3f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:48.450340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83216f58-419d-4def-8f8b-bc7f07f1d3f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:48.450604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83216f58-419d-4def-8f8b-bc7f07f1d3f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:48.459843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83216f58-419d-4def-8f8b-bc7f07f1d3f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:48.461720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83216f58-419d-4def-8f8b-bc7f07f1d3f3 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:48.471137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 611/1213] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:37:50.688299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34321f09-7a8e-4921-a6b9-46223dd597ab None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:50.688872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34321f09-7a8e-4921-a6b9-46223dd597ab None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:50.688872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34321f09-7a8e-4921-a6b9-46223dd597ab None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:50.690227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 603/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:37:50.696754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f407eb-e0bd-4119-ac15-4a6a0fd52b20 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:50.701766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f407eb-e0bd-4119-ac15-4a6a0fd52b20 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:50.701766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f407eb-e0bd-4119-ac15-4a6a0fd52b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:50.719045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a2f407eb-e0bd-4119-ac15-4a6a0fd52b20 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:37:50.720267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a2f407eb-e0bd-4119-ac15-4a6a0fd52b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:50.732178 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 612/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 14:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:37:51.040934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34321f09-7a8e-4921-a6b9-46223dd597ab None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:37:51.042721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-34321f09-7a8e-4921-a6b9-46223dd597ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:37:51.057316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34321f09-7a8e-4921-a6b9-46223dd597ab None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:37:51.072658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47c2c0-8275-4c17-b75e-9c58ae42e9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:37:51.072658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c2c0-8275-4c17-b75e-9c58ae42e9bc service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:37:51.072658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c2c0-8275-4c17-b75e-9c58ae42e9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:37:51.072658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c2c0-8275-4c17-b75e-9c58ae42e9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:37:51.073229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47c2c0-8275-4c17-b75e-9c58ae42e9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:37:51.073229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47c2c0-8275-4c17-b75e-9c58ae42e9bc service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:37:51.083002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 604/1216] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:38:00.713175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=47 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:38:01.078484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=14 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:13.300666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.301661 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.302100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.330964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.332497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.341432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 613/1217] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.353833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.354464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.354771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.407183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.409395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.432536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 605/1218] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.437773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:13.451456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af257-95c1-40b5-8483-f6763ab889a6 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:13.464717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:13.465796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.466314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.466809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.467706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:13.468235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:13.470775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:13.492295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 614/1219] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:13.508608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:13.523605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9d8f8-17f5-4fa2-8d6f-45385d383b34 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:13.533840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590484b-2d75-48b1-b0dc-0ad33f25b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:13.534495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590484b-2d75-48b1-b0dc-0ad33f25b212 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.534769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590484b-2d75-48b1-b0dc-0ad33f25b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.535036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590484b-2d75-48b1-b0dc-0ad33f25b212 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.535674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590484b-2d75-48b1-b0dc-0ad33f25b212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:13.537898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590484b-2d75-48b1-b0dc-0ad33f25b212 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:13.684050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 606/1220] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:13.689459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:13.699983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b046b7-8256-490a-83d0-f440c475bfa3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:13.776780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3341-8600-4a37-95ba-7db1643ca41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:13.777431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3341-8600-4a37-95ba-7db1643ca41e admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.777711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3341-8600-4a37-95ba-7db1643ca41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.777980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3341-8600-4a37-95ba-7db1643ca41e admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.778615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb3341-8600-4a37-95ba-7db1643ca41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:13.780565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb3341-8600-4a37-95ba-7db1643ca41e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:13.803235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3e56-c015-4c23-bbdb-ef7c0f8abe3f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.806566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3e56-c015-4c23-bbdb-ef7c0f8abe3f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.806566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3e56-c015-4c23-bbdb-ef7c0f8abe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.841332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ddef3e56-c015-4c23-bbdb-ef7c0f8abe3f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.847359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ddef3e56-c015-4c23-bbdb-ef7c0f8abe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.860263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 607/1221] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.862490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0cf2-8afe-485e-b2a6-825e3d74586d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.862776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0cf2-8afe-485e-b2a6-825e3d74586d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.863045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0cf2-8afe-485e-b2a6-825e3d74586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.887991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b5ec0cf2-8afe-485e-b2a6-825e3d74586d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.888213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 615/1222] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:13.890992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b5ec0cf2-8afe-485e-b2a6-825e3d74586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.891162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066f8dc-f3da-4a9b-8383-3b687765d5e5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.891162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066f8dc-f3da-4a9b-8383-3b687765d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.891162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066f8dc-f3da-4a9b-8383-3b687765d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.898893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 608/1223] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.900929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1bb3e-ab4c-48dd-a6c4-1c545d473b8f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.901440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1bb3e-ab4c-48dd-a6c4-1c545d473b8f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.901945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1bb3e-ab4c-48dd-a6c4-1c545d473b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.919732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-6066f8dc-f3da-4a9b-8383-3b687765d5e5 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.920933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6066f8dc-f3da-4a9b-8383-3b687765d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.935355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 616/1224] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.935355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.935355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.935355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.936279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-76b1bb3e-ab4c-48dd-a6c4-1c545d473b8f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.937710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76b1bb3e-ab4c-48dd-a6c4-1c545d473b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.951051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 609/1225] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.953428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccb459-d085-417b-8f7f-23c249b0777e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.954092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccb459-d085-417b-8f7f-23c249b0777e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.955039 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccb459-d085-417b-8f7f-23c249b0777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.957204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.958531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.967209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 617/1226] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.968045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:13.978526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-55ccb459-d085-417b-8f7f-23c249b0777e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:13.980870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-55ccb459-d085-417b-8f7f-23c249b0777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:13.984303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e42bf-0fb3-4494-b26a-7ac0bdd0d1ac None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:13.990439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc2696-e0f6-4dfe-9da7-c2bbc1e34038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:13.991048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2696-e0f6-4dfe-9da7-c2bbc1e34038 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.991179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 610/1227] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:13.991433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2696-e0f6-4dfe-9da7-c2bbc1e34038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.991615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2696-e0f6-4dfe-9da7-c2bbc1e34038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.992205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc2696-e0f6-4dfe-9da7-c2bbc1e34038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:13.992380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa5546-798f-48e8-9125-37c8f7bcc2b4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:13.992687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa5546-798f-48e8-9125-37c8f7bcc2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:13.992951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa5546-798f-48e8-9125-37c8f7bcc2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:13.994010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc2696-e0f6-4dfe-9da7-c2bbc1e34038 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.006040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 618/1228] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:14.007843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.008347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.008850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.014349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-71aa5546-798f-48e8-9125-37c8f7bcc2b4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.015534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-71aa5546-798f-48e8-9125-37c8f7bcc2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.025965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 611/1229] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.030368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.030812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bee350-8a09-45e9-91eb-69af507215dd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.031094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bee350-8a09-45e9-91eb-69af507215dd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.031375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bee350-8a09-45e9-91eb-69af507215dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.031573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.043803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 619/1230] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.043803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.051919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343929bc-7a3f-4b52-95a8-76db0cc4d1b1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.056758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.057394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.057627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-48bee350-8a09-45e9-91eb-69af507215dd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.057754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.057953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.058567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:14.058817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-48bee350-8a09-45e9-91eb-69af507215dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.058946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.060705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065185-be1e-42aa-a1c8-d8ff0b3f2657 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.075456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 612/1231] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.077184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.077623 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.077901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.078273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 620/1232] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:14.080133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.080660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.080943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.101805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.102331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.102999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.103779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.111146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 613/1233] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.112095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.114285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 621/1234] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.115335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.123555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624a2cc-44c9-4a73-b764-cac86a0dd350 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.127626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f6789-ef93-4516-9ff5-2f194ea4f2b3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.128756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.129445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.129700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.129972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.130641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.132256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.132395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.132846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.133165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.133492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.134130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:14.134432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.136003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965bcb2-8153-442a-b56d-495c84e727d1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.144090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 614/1235] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:14.145091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.148775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 622/1236] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:14.150032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.150318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.150587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.162254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350953-b043-4763-81a6-be38c755cd76 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.167140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.167784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.168054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.168321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.168952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:14.169283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.180626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:39:14.180626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:39:14.180626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:39:14.180955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.181083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.181173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.181173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 623/1237] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.181925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.189998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 615/1238] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:14.189998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.195727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed58d8-dfab-493b-b1bb-5eef9c90e842 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.197209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1e52f-5d06-479f-a27c-9c9bd2c1e9d1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.200327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.200954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.201261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.201528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.201971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.202220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:14.202566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.202694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.202879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.203168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.203795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.204151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.205756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.220365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 624/1239] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:14.221062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.231153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c0bdb-689d-4b70-816e-aaa7a78d274e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.235907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.236513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.236792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.237105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.237747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:14.238047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.239635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff4c3a-9679-49a5-863d-0ea390ab0e71 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.252296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 625/1240] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:14.253319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.253604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.253876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.274289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.275935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.283302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 626/1241] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.284040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.293026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a941e-1708-4eba-b7b9-0fc49359746f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.298502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04be9a-7ecf-4fb0-ac3d-26855118b694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.298502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04be9a-7ecf-4fb0-ac3d-26855118b694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.298502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04be9a-7ecf-4fb0-ac3d-26855118b694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.298816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04be9a-7ecf-4fb0-ac3d-26855118b694 admin admin] PATH_INFO: `/v3/projects/42718454692446bbb6605fc53d3bae28/users/6fceca70c37a47af88d809d2a1425ca4/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.312975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04be9a-7ecf-4fb0-ac3d-26855118b694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42718454692446bbb6605fc53d3bae28, user_id=6fceca70c37a47af88d809d2a1425ca4, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.316360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04be9a-7ecf-4fb0-ac3d-26855118b694 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.377353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 616/1242] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:14.378139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.440105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc4d43-ee69-490b-a130-f13746181dca admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.445103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.445716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.445988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.446286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.446929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.448838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.483975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 627/1243] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:14 2022] PUT /identity/v3/projects/42718454692446bbb6605fc53d3bae28/users/6fceca70c37a47af88d809d2a1425ca4/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:14.485376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.485845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.486321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.494061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:14.507112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:14.528750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:14.559520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 617/1244] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/projects => generated 391 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:14.560592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.568668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 628/1245] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:14.569749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:14.579159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e7af1-139b-4f8d-aa86-02e3a73cb0fc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.593768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:14.630249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be69873-1f8f-4ad5-8b0e-a81c7d5416a1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:14.635304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.635918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.636239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.636513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.637172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.638977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:14.639108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:14.639582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:14.639861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:14.640131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:14.640759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:14.641847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:39:14.641847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:39:14.641847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:39:14.643174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.073792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 629/1246] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/projects => generated 373 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.074007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 618/1247] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:14 2022] POST /identity/v3/projects => generated 403 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.074517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.077892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.137475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f268-0f96-4588-8b4e-5304b0ba2343 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.141317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f31486-d4bc-4cf8-9807-9e727f610784 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.142129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:15.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.146551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62637962-f4a1-4858-a760-a5228b885099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.146670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62637962-f4a1-4858-a760-a5228b885099 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.146877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62637962-f4a1-4858-a760-a5228b885099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.147184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62637962-f4a1-4858-a760-a5228b885099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.147822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62637962-f4a1-4858-a760-a5228b885099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.149751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62637962-f4a1-4858-a760-a5228b885099 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.158237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 619/1248] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:15.158975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.168504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072aaf6-9af4-470e-8a87-5068cda45d06 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.173666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.174264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.174592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.174831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.175451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.176496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:39:15.176496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:39:15.176496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:39:15.177853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.302594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 630/1249] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/projects => generated 371 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.303901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.304180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.304448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.328962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:15.334988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] MFA Rules not processed for user `6fceca70c37a47af88d809d2a1425ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:15.334988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:15.403445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 631/1250] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:15.404448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.468065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d093e4-5dc2-481b-ac91-eb3073dde392 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.473740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.474345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.474619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.474888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.475509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.477463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.479561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 620/1251] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/users => generated 290 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.480651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.492371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb54ba6-8c20-47df-8ecd-1aae399d075f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.501201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.501201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.501201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.501201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.502450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:15.502450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.502450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.513100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 621/1252] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:15.514018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.524294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058b5a0-a777-4c92-9229-33fdc693754e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.528884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.529677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.529959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.530230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.530839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.533434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.599570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 632/1253] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/users => generated 296 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.600413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.610709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64fd6-5df5-4598-b3b4-0561f21123a9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.615419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.616026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.616301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.616611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.617273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.619304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.639338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 622/1254] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.639338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.650224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da689952-6955-40fb-8c7e-d8ed3334de02 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.654900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.655483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.655754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.656022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.656633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.659406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.747382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 633/1255] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.748310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.758497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e085-cac8-4318-bd12-6711289b09d2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.763388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.763954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.764231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.764490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.765135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.767618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.785817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 623/1256] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.785817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.818199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:15.838933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd57f-737b-469e-8322-7906cb3e955e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.842626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.843220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.843496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.843764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.862137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 634/1257] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:15.863074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.873033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5634f-11ef-447f-a732-fd53fbd98509 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.904803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.906603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.916213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 624/1258] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:15.917132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.926021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:15.932294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.932992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.933404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.933701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.934416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.936063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.949767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 635/1259] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:15.949767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.958287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6b0e-eec5-42d3-9129-4794a364b68e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.964320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.964929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.965241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.965514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.966155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:15.967765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:15.981021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 636/1260] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:15.981909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:15.991787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2304b5-7ca7-4b3d-b9ae-1230b1d7c464 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.995001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad727b-7853-4b56-b0f4-71708d7ab20b service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:15.996927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:15.997608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:15.997833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:15.998108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:15.998741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.000134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.000320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.000735 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.001620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.001895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.002525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.004529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.013908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 637/1261] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.017143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.024756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e534b-8d4b-4995-959a-000c21573b01 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.029687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.030361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.030651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.031266 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.032147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.034150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.045609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 638/1262] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.045609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.055660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbf38a-cc90-4f04-a546-2e9c97b7dd31 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.060728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.061348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.061625 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.061896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.062520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.064580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.097171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 625/1263] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:15 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:16.100721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.161283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc56c-fa69-4609-9cd5-4790c9b81457 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.166281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.167024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.167295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.167561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.168191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.169836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.180400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 626/1264] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.181413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.190850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ac3be-6639-47b5-8528-cab8e470cc20 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.195471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.196052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.196330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.196597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.197251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.198829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.200822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:16.209991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 627/1265] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.210825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.220494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebab56a-ea8f-40bd-8ae5-18fbc159da27 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.222580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 639/1266] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:16 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:16.229071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.229253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.229253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.229253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.229253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.229639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.229639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.235272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884fb7d-4820-44c3-bedd-4f3a20e7195f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.243495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 628/1267] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.243652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.243770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.243873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.243873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.243873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.243873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.244492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.255868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cc0d2-90ae-4177-bc55-c40da801c043 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.256709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 640/1268] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.257586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.264707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.266072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.266872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.267319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.267643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df480f0-f966-41a9-929b-552610202ae3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.268401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.272040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.272515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.273175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.273483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.273745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.274383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.276276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.282659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 629/1269] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.295685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.295685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065d6cf-51b7-49ed-bd00-8d59575bae53 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.299341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.299931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.300210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.300482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.301147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.302732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.315586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 630/1270] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.316619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.325899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12dc97-0442-4896-9877-b77a9d1a4a64 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.330510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.331123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.331462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.331729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] PATH_INFO: `/v3/projects/b60403061fba42eba98a7c89987672a8/users/a8706e96002946b2b9bfe87de5c3271a/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.346306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60403061fba42eba98a7c89987672a8, user_id=a8706e96002946b2b9bfe87de5c3271a, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.349096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.449928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 641/1271] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:16 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:16.450598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.460687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d2b80-34e8-4c60-8543-30252e9dc062 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.465434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.466055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.466333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.466603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.467231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.468818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.480151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 642/1272] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:16.480933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.491500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e3896-2c6d-4725-beca-57ce05c24e4f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.493114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 631/1273] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:16 2022] PUT /identity/v3/projects/b60403061fba42eba98a7c89987672a8/users/a8706e96002946b2b9bfe87de5c3271a/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:16.493114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.552835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.553489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.553763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.554027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] PATH_INFO: `/v3/projects/edc675d4a92f4ff9ae8a41a020fa792c/users/921ba33e2e0740278b7b6fb4af248363/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.558110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7def-7d14-4c99-a80b-dde93820a1eb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.565461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.566238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.566517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.566809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] PATH_INFO: `/v3/projects/eb9e9706da874e1cb92bc199af1053da/users/899392e0a3c343bc87fe4283a1674b63/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.571355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc675d4a92f4ff9ae8a41a020fa792c, user_id=921ba33e2e0740278b7b6fb4af248363, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.573459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.582680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9e9706da874e1cb92bc199af1053da, user_id=899392e0a3c343bc87fe4283a1674b63, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.585393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.669998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 643/1274] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:16 2022] PUT /identity/v3/projects/edc675d4a92f4ff9ae8a41a020fa792c/users/921ba33e2e0740278b7b6fb4af248363/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:16.670811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.712453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 632/1275] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:16 2022] PUT /identity/v3/projects/eb9e9706da874e1cb92bc199af1053da/users/899392e0a3c343bc87fe4283a1674b63/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:16.713356 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.728783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:16.734467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69511c-9713-4ab0-92ef-8efd76d9da20 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.785902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9fbba-3dc3-4c8a-8ee7-7e1fb0755145 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.791034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.791702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.791983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.792250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] PATH_INFO: `/v3/projects/a8912867218d4b1cb7de0479b9194ea9/users/558b2e2e4ee54db1a040bd2e233b53d9/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.794887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.796471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:16.796749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:16.797033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] PATH_INFO: `/v3/projects/60c52c5ab5864f5094b8c170cbb9f81c/users/0e97852d54174b04a15fe45286d19879/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:16.810750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8912867218d4b1cb7de0479b9194ea9, user_id=558b2e2e4ee54db1a040bd2e233b53d9, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.812376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c52c5ab5864f5094b8c170cbb9f81c, user_id=0e97852d54174b04a15fe45286d19879, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:16.814655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.816004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:16.897795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 644/1277] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:16 2022] PUT /identity/v3/projects/60c52c5ab5864f5094b8c170cbb9f81c/users/0e97852d54174b04a15fe45286d19879/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:16.898638 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.912238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 633/1277] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:16 2022] PUT /identity/v3/projects/a8912867218d4b1cb7de0479b9194ea9/users/558b2e2e4ee54db1a040bd2e233b53d9/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:16.912238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:16.924989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:16.984339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d792d8-9f2a-4b6b-aa1f-4e50412ce3a4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.986811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd89ac-c0b3-4ac4-9334-72b5e04704d7 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:16.991496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68d1a2-2987-4986-a0b8-059282ad8fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.991943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9abe8-c442-4d56-9e82-e9d73f628373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:16.992505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9abe8-c442-4d56-9e82-e9d73f628373 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.002186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9abe8-c442-4d56-9e82-e9d73f628373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.002351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68d1a2-2987-4986-a0b8-059282ad8fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.002442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9abe8-c442-4d56-9e82-e9d73f628373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.002521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68d1a2-2987-4986-a0b8-059282ad8fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.002521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68d1a2-2987-4986-a0b8-059282ad8fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.002677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9abe8-c442-4d56-9e82-e9d73f628373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.002768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68d1a2-2987-4986-a0b8-059282ad8fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.002852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9abe8-c442-4d56-9e82-e9d73f628373 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.002940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68d1a2-2987-4986-a0b8-059282ad8fa1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.008159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 634/1278] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:17.009789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66891e-e2f0-4337-8ea8-1092e7a19f7f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.010460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66891e-e2f0-4337-8ea8-1092e7a19f7f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.010962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66891e-e2f0-4337-8ea8-1092e7a19f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.012323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 645/1279] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:17.013727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26709660-5543-4160-b2fa-d47ec5ec6c11 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.014180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26709660-5543-4160-b2fa-d47ec5ec6c11 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.014516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26709660-5543-4160-b2fa-d47ec5ec6c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.035747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fc66891e-e2f0-4337-8ea8-1092e7a19f7f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.037129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fc66891e-e2f0-4337-8ea8-1092e7a19f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.039212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-26709660-5543-4160-b2fa-d47ec5ec6c11 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.040394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-26709660-5543-4160-b2fa-d47ec5ec6c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.046270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 635/1280] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:17.047316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359de283-e18f-4614-93e2-0f21d2e36d7c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.048723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359de283-e18f-4614-93e2-0f21d2e36d7c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.048938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359de283-e18f-4614-93e2-0f21d2e36d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.052380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 646/1281] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:17.053725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.053988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.054291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.085492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-359de283-e18f-4614-93e2-0f21d2e36d7c None None] MFA Rules not processed for user `899392e0a3c343bc87fe4283a1674b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.085635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] MFA Rules not processed for user `558b2e2e4ee54db1a040bd2e233b53d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.086760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.086883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-359de283-e18f-4614-93e2-0f21d2e36d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.159979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:17.162736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 636/1282] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:17.164016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.164305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.164580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.170227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 647/1283] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:17.171239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.190093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c1a9-3402-4452-a29b-e2e6f87507d0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.196044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.196748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.197087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.197364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] PATH_INFO: `/v3/projects/0dd67ec4f08d4d548af5b5ccee50407c/users/eb0b112497dc4b64a9ea2f09f808646d/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.203750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] MFA Rules not processed for user `0e97852d54174b04a15fe45286d19879`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.205382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.213854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd67ec4f08d4d548af5b5ccee50407c, user_id=eb0b112497dc4b64a9ea2f09f808646d, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.217417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.273373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 637/1284] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:17.274302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.288154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680de4f-0b38-480b-a118-f69df2836b4b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.296524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 648/1285] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:17 2022] PUT /identity/v3/projects/0dd67ec4f08d4d548af5b5ccee50407c/users/eb0b112497dc4b64a9ea2f09f808646d/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:17.297409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.305851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:17.353397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.354016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.354289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.354556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.355193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.356853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.370490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 638/1286] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:17.370490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.374992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63af5c-3e11-4a1b-8443-d6cfc1da3a61 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.378575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a66195-3562-4622-ac87-85ae99a78fa1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.379071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.379685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.379955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.380271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.382448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.383037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.383925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.383925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.390111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.391730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.392624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.394064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.401239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 649/1287] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:17.402180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.404261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 639/1288] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:17.405501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.412832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a63b26-dfca-4d0c-a48e-3b3aac4d9425 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.418265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36579047-0dda-4c8a-8173-845197d5ad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.418549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4746213-5e86-47fb-8c87-664c27557987 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.419283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579047-0dda-4c8a-8173-845197d5ad81 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.419624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579047-0dda-4c8a-8173-845197d5ad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.419961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579047-0dda-4c8a-8173-845197d5ad81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.420703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36579047-0dda-4c8a-8173-845197d5ad81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.423619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36579047-0dda-4c8a-8173-845197d5ad81 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.424808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82b8c5-1c94-4922-9c73-59789ee76f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.425461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82b8c5-1c94-4922-9c73-59789ee76f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.426017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82b8c5-1c94-4922-9c73-59789ee76f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.426362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82b8c5-1c94-4922-9c73-59789ee76f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.427210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82b8c5-1c94-4922-9c73-59789ee76f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.429648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82b8c5-1c94-4922-9c73-59789ee76f29 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.581868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 640/1289] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:17.582740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 650/1290] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/projects => generated 405 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:17.583836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d69080-154e-4f0f-8e69-32d7fdf90e54 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.584123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d69080-154e-4f0f-8e69-32d7fdf90e54 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.584398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d69080-154e-4f0f-8e69-32d7fdf90e54 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.584513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.584680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.585044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.589157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 641/1291] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 14:39:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:17.589960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d69080-154e-4f0f-8e69-32d7fdf90e54 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.612916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.614204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.682020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d69080-154e-4f0f-8e69-32d7fdf90e54 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.687284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.688082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.688421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.688695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] PATH_INFO: `/v3/projects/215b01dc4fa14b1da39498761f0018de/users/c45cd5543178463f85a7c7f062389a5f/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.696696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 651/1292] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:17.697393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.707499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af886c-32b8-4438-aae3-ecf5b7eb9bd7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.708175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=215b01dc4fa14b1da39498761f0018de, user_id=c45cd5543178463f85a7c7f062389a5f, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.711073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.712238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d9cc7-2359-4f9b-9a92-d0870f3b1ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.712840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d9cc7-2359-4f9b-9a92-d0870f3b1ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.713158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d9cc7-2359-4f9b-9a92-d0870f3b1ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.713435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d9cc7-2359-4f9b-9a92-d0870f3b1ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.714051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d9cc7-2359-4f9b-9a92-d0870f3b1ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.715942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d9cc7-2359-4f9b-9a92-d0870f3b1ae6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.816975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 642/1293] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:17 2022] PUT /identity/v3/projects/215b01dc4fa14b1da39498761f0018de/users/c45cd5543178463f85a7c7f062389a5f/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:17.826995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:17.887206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 652/1294] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:17.887206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92783887-db12-4307-98de-bd053eec1552 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.887206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92783887-db12-4307-98de-bd053eec1552 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.887206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92783887-db12-4307-98de-bd053eec1552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.932994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5e3c-9a7b-4a47-89d9-490a5fb04673 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:17.933258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-92783887-db12-4307-98de-bd053eec1552 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:17.934620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-92783887-db12-4307-98de-bd053eec1552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:17.937784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e182fa4-6922-4486-80e2-52fa41ca5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:17.938953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e182fa4-6922-4486-80e2-52fa41ca5ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:17.939220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e182fa4-6922-4486-80e2-52fa41ca5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:17.939488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e182fa4-6922-4486-80e2-52fa41ca5ecd admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:17.940109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e182fa4-6922-4486-80e2-52fa41ca5ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:17.942075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e182fa4-6922-4486-80e2-52fa41ca5ecd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:17.950126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:17.952978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:18.003534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 653/1295] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:18.004386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92783887-db12-4307-98de-bd053eec1552 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.056258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 643/1296] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:17 2022] POST /identity/v3/users => generated 309 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:18.057927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.063030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.063030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.070775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92783887-db12-4307-98de-bd053eec1552 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.078242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.080673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.080911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.081250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.100484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.101970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.115304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 654/1297] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:18.116361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] MFA Rules not processed for user `c45cd5543178463f85a7c7f062389a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:18.117693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:18.118064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.130897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b0af7-52e7-4fce-bfe9-660117d92630 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.140600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.140600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.140600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.140600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.141073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.141073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.162215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 655/1298] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:18.162215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.190599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21597-3b0c-412e-b1aa-ac5cc86d2199 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.201724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.202799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.207411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.207411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.207704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 644/1299] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:18.207704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.221752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5e663-1c7c-4310-80c8-955fd3c328e6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.228465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.229672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.229672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.229893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.231725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.233177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.257083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 645/1300] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:18.258033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.270457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2578b9c-46c2-4b3b-9c46-3cb5df27f1c3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.276706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.277421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.277699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.277971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] PATH_INFO: `/v3/projects/48cb09ee47f744abae480beb4388fdcb/users/b145e51831aa4adca3e7fd2eb6fc2dc5/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.287826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.289474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.293194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cb09ee47f744abae480beb4388fdcb, user_id=b145e51831aa4adca3e7fd2eb6fc2dc5, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.297614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.305345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 656/1301] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:18.306304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.318211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcf30f-0e80-4834-af2a-8d53cbee2ba9 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.323188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.323797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.324184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.324466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.325158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.329818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.455618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 646/1302] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:18 2022] PUT /identity/v3/projects/48cb09ee47f744abae480beb4388fdcb/users/b145e51831aa4adca3e7fd2eb6fc2dc5/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:18.455618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.455871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 657/1303] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:18 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:18.456102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.519873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:18.579922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a968dc-ed94-4919-982f-d9a21a0b74aa admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.585799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2c055-5b3c-4cf3-b573-e3ac102c5d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.586484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2c055-5b3c-4cf3-b573-e3ac102c5d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.586743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2c055-5b3c-4cf3-b573-e3ac102c5d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.587110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2c055-5b3c-4cf3-b573-e3ac102c5d32 admin admin] PATH_INFO: `/v3/projects/e2697e25a08d45d7b167df89d62a9b0e/users/9f7fe780350a46a183dcdefa8361fa42/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.591232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553db45-cce0-445a-9907-86af5a127dba admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.599262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.604138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.604138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.604138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.604138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.604138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.604683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2c055-5b3c-4cf3-b573-e3ac102c5d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2697e25a08d45d7b167df89d62a9b0e, user_id=9f7fe780350a46a183dcdefa8361fa42, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.607311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2c055-5b3c-4cf3-b573-e3ac102c5d32 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.616716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 658/1304] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:39:18.617582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.627496 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa0bbc-162b-4656-8f9e-4e2ab9dba747 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.632511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.638852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.638852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.638852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.638852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.638852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.718714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 647/1305] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:18 2022] PUT /identity/v3/projects/e2697e25a08d45d7b167df89d62a9b0e/users/9f7fe780350a46a183dcdefa8361fa42/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:18.718714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a99f-5bba-452d-a0f7-51e7333172cb None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.718714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a99f-5bba-452d-a0f7-51e7333172cb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.718714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a99f-5bba-452d-a0f7-51e7333172cb None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.718714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 648/1306] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:18.719473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a99f-5bba-452d-a0f7-51e7333172cb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.754282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:18.776740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 659/1307] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:18 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:18.777909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.791707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4f2fa-530c-4582-893d-9b2fd00f4433 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.817033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a99f-5bba-452d-a0f7-51e7333172cb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.826938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8b188-47a7-4db0-a439-276af43cfe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.827612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8b188-47a7-4db0-a439-276af43cfe74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.834701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8b188-47a7-4db0-a439-276af43cfe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.834701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8b188-47a7-4db0-a439-276af43cfe74 admin admin] PATH_INFO: `/v3/projects/48cb09ee47f744abae480beb4388fdcb/users/b145e51831aa4adca3e7fd2eb6fc2dc5/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.837831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8b188-47a7-4db0-a439-276af43cfe74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cb09ee47f744abae480beb4388fdcb, user_id=b145e51831aa4adca3e7fd2eb6fc2dc5, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.840522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8b188-47a7-4db0-a439-276af43cfe74 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.869092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 649/1308] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:18 2022] PUT /identity/v3/projects/48cb09ee47f744abae480beb4388fdcb/users/b145e51831aa4adca3e7fd2eb6fc2dc5/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:18.870777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb93d41-32ac-41d3-bf7f-d1fdfadaf0e7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.871142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb93d41-32ac-41d3-bf7f-d1fdfadaf0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.871512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb93d41-32ac-41d3-bf7f-d1fdfadaf0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.897517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eeb93d41-32ac-41d3-bf7f-d1fdfadaf0e7 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:18.897822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.897822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.897822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.897822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.898188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eeb93d41-32ac-41d3-bf7f-d1fdfadaf0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:18.898281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.898281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:18.908397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 660/1309] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:18.909279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:18.919408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b69ee-ca24-42ba-b1e1-dd616d03dc25 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:18.982733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 650/1310] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:39:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:18.983995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.984230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.984504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.989145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:18.989894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:18.990368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:18.990996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:18.991415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:18.993981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.008551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 661/1311] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:19.008551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:19.021641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] MFA Rules not processed for user `b145e51831aa4adca3e7fd2eb6fc2dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:19.023094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:19.023481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d073a-3432-4752-8474-b25e1b5c1687 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:19.028523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:19.029214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.029495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.029767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] PATH_INFO: `/v3/projects/e2697e25a08d45d7b167df89d62a9b0e/users/9f7fe780350a46a183dcdefa8361fa42/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.032283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2697e25a08d45d7b167df89d62a9b0e, user_id=9f7fe780350a46a183dcdefa8361fa42, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:19.035174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.047307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 662/1312] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:19 2022] PUT /identity/v3/projects/e2697e25a08d45d7b167df89d62a9b0e/users/9f7fe780350a46a183dcdefa8361fa42/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:19.048554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:19.073327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:19.104871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 651/1313] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:19.106004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:19.124672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdb706-0f2e-4b52-a846-f13fa238b71a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:19.162969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a60516-d692-49d2-b5c6-e37b0050a96c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:19.170103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ad15e-f6df-4fdd-bd4d-0dd1a7be532a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:19.171132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ad15e-f6df-4fdd-bd4d-0dd1a7be532a service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.171993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ad15e-f6df-4fdd-bd4d-0dd1a7be532a service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.172289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ad15e-f6df-4fdd-bd4d-0dd1a7be532a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.214939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:19.215683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.216104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.216437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] PATH_INFO: `/v3/projects/ff21a7c295b44bb29782029696e1b7bd/users/34f40786d942450ea55c5f35c3f85fb6/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.234753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff21a7c295b44bb29782029696e1b7bd, user_id=34f40786d942450ea55c5f35c3f85fb6, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:19.238509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.284981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ad15e-f6df-4fdd-bd4d-0dd1a7be532a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:19.286537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ad15e-f6df-4fdd-bd4d-0dd1a7be532a service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.304567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 663/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:19.306086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.306379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.306657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.347594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 652/1315] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:19 2022] PUT /identity/v3/projects/ff21a7c295b44bb29782029696e1b7bd/users/34f40786d942450ea55c5f35c3f85fb6/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:19.347594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:19.374768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] MFA Rules not processed for user `9f7fe780350a46a183dcdefa8361fa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:19.377376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:19.467730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:19.510845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6a25e-ab7c-41d1-a800-7d7b02cf1645 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:19.520486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d5f2d-22bb-4b50-bd3a-900d8f0906aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:19.521136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d5f2d-22bb-4b50-bd3a-900d8f0906aa service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.521453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d5f2d-22bb-4b50-bd3a-900d8f0906aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.524272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d5f2d-22bb-4b50-bd3a-900d8f0906aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.585509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 664/1316] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:19.587438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:19.709244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d5f2d-22bb-4b50-bd3a-900d8f0906aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:19.714868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d5f2d-22bb-4b50-bd3a-900d8f0906aa service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.740516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 653/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:19.744146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611dac-f898-4f53-909c-b9a58d9bdedf None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.746193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611dac-f898-4f53-909c-b9a58d9bdedf None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.746193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611dac-f898-4f53-909c-b9a58d9bdedf None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.750560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 654/1318] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 14:39:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:19.754770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5add-6dd7-4c62-b910-98519e1c7557 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.755062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5add-6dd7-4c62-b910-98519e1c7557 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.755341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5add-6dd7-4c62-b910-98519e1c7557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.770032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8f9e-ee82-4dec-baf2-14538ebf6f2b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:19.789622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:19.790272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.792678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.792979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.793686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:19.800360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.811261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ccfd5add-6dd7-4c62-b910-98519e1c7557 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:19.812757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ccfd5add-6dd7-4c62-b910-98519e1c7557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:19.830660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 665/1319] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:19.836282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:19.854060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327cd9e-aa98-4809-bab3-ddf055faf275 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:19.862791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd97e48-2c75-4bc7-b618-223c538384f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:19.864088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd97e48-2c75-4bc7-b618-223c538384f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.864475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd97e48-2c75-4bc7-b618-223c538384f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.864923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd97e48-2c75-4bc7-b618-223c538384f9 admin admin] PATH_INFO: `/v3/projects/ff21a7c295b44bb29782029696e1b7bd/users/34f40786d942450ea55c5f35c3f85fb6/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.867550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd97e48-2c75-4bc7-b618-223c538384f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff21a7c295b44bb29782029696e1b7bd, user_id=34f40786d942450ea55c5f35c3f85fb6, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:19.872406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd97e48-2c75-4bc7-b618-223c538384f9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:19.901055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 666/1320] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:19 2022] PUT /identity/v3/projects/ff21a7c295b44bb29782029696e1b7bd/users/34f40786d942450ea55c5f35c3f85fb6/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:19.911974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.914621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.915337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.956582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 655/1321] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:19.976893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7ea0c-fbe5-4f1f-a5f3-92a780572fc8 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.978576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7ea0c-fbe5-4f1f-a5f3-92a780572fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.979013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7ea0c-fbe5-4f1f-a5f3-92a780572fc8 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:19.980518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] MFA Rules not processed for user `34f40786d942450ea55c5f35c3f85fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:19.981918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:19.983018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 656/1322] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 14:39:19 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:19.989852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:19.990407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:19.990862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:20.030997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:20.035719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:20.088243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 667/1323] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:20.089693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:20.169195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a95bd3-1cb2-415e-9fb1-9b7e66389332 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:20.175169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:20.176119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:20.176629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:20.177201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:20.184797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 657/1324] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:20.234724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:20.269628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:20.271622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:20.285753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 668/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:20.490178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:20.500119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afd96f-a246-45cc-b324-9153dc209cb9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:20.505195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:20.506206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:20.506206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:20.506412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:20.517225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:20.518962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:20.530266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 658/1326] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:20.631009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:20.643790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1359af-6c45-4bd0-b415-344ba9f6904e service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:20.648478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:20.649147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:20.649391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:20.649789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:20.662281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:20.664957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:20.683380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 669/1327] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:21.232957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:21.319649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:21.451931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:21.457406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:21.471325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7df9f-d7be-4ad4-bb4e-86d03997a8db service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:21.480110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547114-bacf-4321-b43a-1873176b9d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:21.480657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547114-bacf-4321-b43a-1873176b9d7e service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:21.480933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547114-bacf-4321-b43a-1873176b9d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:21.489418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547114-bacf-4321-b43a-1873176b9d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:21.636259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f5915-1220-43b9-9533-c17be15df762 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:21.648249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:21.649302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:21.649583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:21.649714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547114-bacf-4321-b43a-1873176b9d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:21.649863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] PATH_INFO: `/v3/users/899392e0a3c343bc87fe4283a1674b63` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:21.651088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=899392e0a3c343bc87fe4283a1674b63)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:21.652161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547114-bacf-4321-b43a-1873176b9d7e service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:21.653335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:21.681402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 670/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:22.013983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899392e0a3c343bc87fe4283a1674b63'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:22.050556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899392e0a3c343bc87fe4283a1674b63'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:22.148555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899392e0a3c343bc87fe4283a1674b63'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:22.154486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899392e0a3c343bc87fe4283a1674b63'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:22.154486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899392e0a3c343bc87fe4283a1674b63'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:22.201870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:22.202590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:22.202590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:22.215521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 659/1329] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:21 2022] DELETE /identity/v3/users/899392e0a3c343bc87fe4283a1674b63 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:22.236934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:22.242960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:22.379976 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 671/1330] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:22.389820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:22.392691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:22.466928 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92cd14-b72c-498f-ab09-550a654a3594 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:22.472630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:22.473520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:22.473903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:22.474278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:22.536247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:22.537768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:22.566039 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 660/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:39:23.285415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:23.445825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:23.448088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:23.471374 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41349db3-2998-4592-9614-3c94e4b43bfd None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:23.484244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:23.485826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:23.486034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:23.486460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] PATH_INFO: `/v3/users/558b2e2e4ee54db1a040bd2e233b53d9` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:23.487924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] RBAC: Authorizing `identity:delete_user(user_id=558b2e2e4ee54db1a040bd2e233b53d9)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:23.489381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: WARNING py.warnings [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:39:23.489381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: warnings.warn(msg) Nov 02 14:39:23.489381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: Nov 02 14:39:23.493880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:23.854552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558b2e2e4ee54db1a040bd2e233b53d9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:23.884639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:23.887021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:23.937718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ded9-c1d6-47b1-a756-b4188b95bb68 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:23.942319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558b2e2e4ee54db1a040bd2e233b53d9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:23.983126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:23.991681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:23.994910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:23.994910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] PATH_INFO: `/v3/projects/eb9e9706da874e1cb92bc199af1053da` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:23.996641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9e9706da874e1cb92bc199af1053da)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:24.007002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: WARNING py.warnings [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 14:39:24.007002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: warnings.warn(msg) Nov 02 14:39:24.007002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: Nov 02 14:39:24.008879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:24.027623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558b2e2e4ee54db1a040bd2e233b53d9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.029226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558b2e2e4ee54db1a040bd2e233b53d9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.030097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d2efd0f4-9bb3-470a-846b-90199c7f037c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558b2e2e4ee54db1a040bd2e233b53d9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.097930 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 672/1332] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:23 2022] DELETE /identity/v3/users/558b2e2e4ee54db1a040bd2e233b53d9 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:24.103802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:24.104166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:24.104450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:24.140429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:24.170186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:24.172242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:24.275320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9e9706da874e1cb92bc199af1053da', 'request_id': 'req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.321492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9e9706da874e1cb92bc199af1053da', 'request_id': 'req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.343131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9e9706da874e1cb92bc199af1053da', 'request_id': 'req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.402580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 673/1333] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:24.403617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:24.411995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:24.443486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] The token cache is being invalidate because project eb9e9706da874e1cb92bc199af1053da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:24.444789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.446562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:24.450111 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 661/1334] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:23 2022] DELETE /identity/v3/projects/eb9e9706da874e1cb92bc199af1053da => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:24.453577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:24.454408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eacda-89d7-41b0-aa51-ca590e4f78a1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:24.457602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:24.642301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e12a1-0fe5-4e08-a32d-c3cd9aa56311 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:24.658807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:24.659505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:24.659978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:24.660589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:24.663808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:24.674930 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:24.675870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:24.676215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:24.681426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] PATH_INFO: `/v3/users/0e97852d54174b04a15fe45286d19879` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:24.682887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e97852d54174b04a15fe45286d19879)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:24.686438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:24.793398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:24.795549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:24.809449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 662/1335] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:24.813994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:24.813994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:24.835337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546cd7-0594-47c5-995d-3832cd6e007b service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:24.854262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:24.855113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:24.855541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:24.855985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] PATH_INFO: `/v3/users/9f7fe780350a46a183dcdefa8361fa42` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:24.857416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7fe780350a46a183dcdefa8361fa42)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:24.859548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:24.965040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e97852d54174b04a15fe45286d19879'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.101857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e97852d54174b04a15fe45286d19879'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.124466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e97852d54174b04a15fe45286d19879'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.127936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e97852d54174b04a15fe45286d19879'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.128620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ea256942-e5b1-4882-bd29-7663e90a2efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e97852d54174b04a15fe45286d19879'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.156216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 674/1336] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:24 2022] DELETE /identity/v3/users/0e97852d54174b04a15fe45286d19879 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:25.160432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:25.160942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:25.161464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:25.208714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:25.210240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:25.219138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7fe780350a46a183dcdefa8361fa42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.244070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7fe780350a46a183dcdefa8361fa42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.339716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 675/1337] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:25.357992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7fe780350a46a183dcdefa8361fa42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.361410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7fe780350a46a183dcdefa8361fa42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.361410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7fe780350a46a183dcdefa8361fa42'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:25.380608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:25.381210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:25.384416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 663/1338] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:24 2022] DELETE /identity/v3/users/9f7fe780350a46a183dcdefa8361fa42 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:25.516615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea17509-3f46-48bf-a2ec-9d9db4565d3e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:25.740881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:25.741486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:25.741765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:25.742038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:25.744975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:25.919733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:25.925697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:25.956038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 676/1339] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:39:26.004260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:26.022057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:26.023759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:26.060184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442739f0-5e62-4eb4-bf57-d47317a9b5f2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:26.078120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:26.079928 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:26.080429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:26.081726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] PATH_INFO: `/v3/projects/a8912867218d4b1cb7de0479b9194ea9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:26.086405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8912867218d4b1cb7de0479b9194ea9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:26.087756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:26.276626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8912867218d4b1cb7de0479b9194ea9', 'request_id': 'req-8c736788-7584-4092-a8f2-d087a71fc68e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:26.305167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8912867218d4b1cb7de0479b9194ea9', 'request_id': 'req-8c736788-7584-4092-a8f2-d087a71fc68e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:26.327374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8912867218d4b1cb7de0479b9194ea9', 'request_id': 'req-8c736788-7584-4092-a8f2-d087a71fc68e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:26.448339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] The token cache is being invalidate because project a8912867218d4b1cb7de0479b9194ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:26.449149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:26.451148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:26.454566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 664/1340] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:26 2022] DELETE /identity/v3/projects/a8912867218d4b1cb7de0479b9194ea9 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:26.459001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:26.472491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135a958-f74b-4ed0-9d2b-90d539da8a84 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:26.635877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:26.642273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:26.642273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:26.642273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] PATH_INFO: `/v3/users/b145e51831aa4adca3e7fd2eb6fc2dc5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:26.642552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b145e51831aa4adca3e7fd2eb6fc2dc5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:26.646730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:27.087989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b145e51831aa4adca3e7fd2eb6fc2dc5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.226688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b145e51831aa4adca3e7fd2eb6fc2dc5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.253093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b145e51831aa4adca3e7fd2eb6fc2dc5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.255893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b145e51831aa4adca3e7fd2eb6fc2dc5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.258528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-78e1440a-3232-4b04-99b8-3ac2de03bbb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b145e51831aa4adca3e7fd2eb6fc2dc5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.318924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 677/1341] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:26 2022] DELETE /identity/v3/users/b145e51831aa4adca3e7fd2eb6fc2dc5 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:27.375666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:27.396294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:27.401341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:27.429373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736788-7584-4092-a8f2-d087a71fc68e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:27.615621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:27.615621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:27.615621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:27.615621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] PATH_INFO: `/v3/projects/e2697e25a08d45d7b167df89d62a9b0e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:27.616551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2697e25a08d45d7b167df89d62a9b0e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:27.616551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:27.804449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2697e25a08d45d7b167df89d62a9b0e', 'request_id': 'req-a863e47a-942a-4435-a879-cc41a3fce5e2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.815403 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2697e25a08d45d7b167df89d62a9b0e', 'request_id': 'req-a863e47a-942a-4435-a879-cc41a3fce5e2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.829509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2697e25a08d45d7b167df89d62a9b0e', 'request_id': 'req-a863e47a-942a-4435-a879-cc41a3fce5e2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.948806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] The token cache is being invalidate because project e2697e25a08d45d7b167df89d62a9b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:27.949129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.950688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a863e47a-942a-4435-a879-cc41a3fce5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:27.958302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 665/1342] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:27 2022] DELETE /identity/v3/projects/e2697e25a08d45d7b167df89d62a9b0e => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:27.959459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:27.959940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:27.960143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:27.994964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:27.996629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:28.096626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 678/1343] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:28.106445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:28.106806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:28.107145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:28.154943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:28.164389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:28.188625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 666/1344] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:28.188843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:28.215873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d4da7-cf6a-461a-9504-4c3592e78bdb None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:28.216730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:28.217641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:28.218159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:28.218652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:28.219903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:28.220756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:28.222642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:28.239778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 679/1345] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:28.245181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:28.256397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59777-14ea-471c-b4ee-c172d88282ed None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:28.267904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:28.268877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:28.269625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:28.270694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:28.271622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:28.273800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:28.457496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 667/1346] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:28 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:28.459353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:28.479198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554e92-4776-4cac-9991-4755a4bb2a0e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:28.655756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:28.660575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:28.678283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:28.679026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:28.679407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:28.680933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:28.681778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:28.692557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:28.868407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 680/1347] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:28 2022] POST /identity/v3/users => generated 283 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:28.872480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:28.898021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea7c09-2e1c-4c69-b700-70cc531ca08b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:28.925738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd92ffd-554b-423b-b446-05f6deb8bf85 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:28.930216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:28.931209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:28.931710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:28.932188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] PATH_INFO: `/v3/projects/60c52c5ab5864f5094b8c170cbb9f81c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:28.934720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c52c5ab5864f5094b8c170cbb9f81c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:28.938152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:28.942176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:28.942807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:28.943089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:28.943551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:28.946537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:28.949281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:28.986320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 681/1348] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:39:28.990323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:29.019820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a2721-24bb-4206-b96c-fda9d5321720 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:29.038391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:29.039535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.040756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.041338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.042152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:29.046823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:29.084496 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 682/1349] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:29.085596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:29.114838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24c71-1c4e-4add-8b89-08033b4a6a5d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:29.128363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11d70e-4b7b-41fc-8c81-64a9f30b75a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:29.130191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d70e-4b7b-41fc-8c81-64a9f30b75a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.130928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d70e-4b7b-41fc-8c81-64a9f30b75a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.132428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d70e-4b7b-41fc-8c81-64a9f30b75a4 admin admin] PATH_INFO: `/v3/projects/5652c00304d64379a5802b52c9f298f5/users/0c1c8080db1249228e6ec4b5a690a9e5/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.168426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c52c5ab5864f5094b8c170cbb9f81c', 'request_id': 'req-f7573f3f-7b34-43e9-91ca-619daff99835'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.169998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11d70e-4b7b-41fc-8c81-64a9f30b75a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5652c00304d64379a5802b52c9f298f5, user_id=0c1c8080db1249228e6ec4b5a690a9e5, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:29.179615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11d70e-4b7b-41fc-8c81-64a9f30b75a4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:29.196863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c52c5ab5864f5094b8c170cbb9f81c', 'request_id': 'req-f7573f3f-7b34-43e9-91ca-619daff99835'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.212158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c52c5ab5864f5094b8c170cbb9f81c', 'request_id': 'req-f7573f3f-7b34-43e9-91ca-619daff99835'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.357185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 683/1350] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:29 2022] PUT /identity/v3/projects/5652c00304d64379a5802b52c9f298f5/users/0c1c8080db1249228e6ec4b5a690a9e5/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:29.367980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.368270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.368559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.391288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:29.395576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] The token cache is being invalidate because project 60c52c5ab5864f5094b8c170cbb9f81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:29.395974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.396852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f7573f3f-7b34-43e9-91ca-619daff99835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.402700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 668/1351] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:28 2022] DELETE /identity/v3/projects/60c52c5ab5864f5094b8c170cbb9f81c => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:29.407999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f5e1-3bda-4380-8a2e-4290958b3194 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.408392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f5e1-3bda-4380-8a2e-4290958b3194 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.410288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] MFA Rules not processed for user `0c1c8080db1249228e6ec4b5a690a9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:29.410912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f5e1-3bda-4380-8a2e-4290958b3194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.417038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:29.466528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-daf0f5e1-3bda-4380-8a2e-4290958b3194 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:29.473599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-daf0f5e1-3bda-4380-8a2e-4290958b3194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:29.541053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 684/1352] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:29.543983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:29.545704 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:29.626842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 669/1353] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:29.629951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.631745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.632215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.681516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:29.683701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:29.694480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 670/1354] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:29.695557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:29.698004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:29.711641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c4dad-9646-4d3a-b700-c2ca4cc7a5b2 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:29.712371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abf259-5352-4a6e-910d-3dbe4dbf625e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:29.717761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:29.718872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.719442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.719950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] PATH_INFO: `/v3/projects/48cb09ee47f744abae480beb4388fdcb` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.720314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:29.721730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cb09ee47f744abae480beb4388fdcb)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:29.721954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:29.722425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:29.722760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:29.724538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:29.725174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:29.758475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:29.909186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 685/1355] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:29 2022] POST /identity/v3/projects => generated 363 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:39:29.909186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:29.920133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cb09ee47f744abae480beb4388fdcb', 'request_id': 'req-61eb139b-6b62-49d1-9662-15846e1c0b74'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.936902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cb09ee47f744abae480beb4388fdcb', 'request_id': 'req-61eb139b-6b62-49d1-9662-15846e1c0b74'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:29.958369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cb09ee47f744abae480beb4388fdcb', 'request_id': 'req-61eb139b-6b62-49d1-9662-15846e1c0b74'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:30.056393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc7967-cb62-494d-a4ca-f64b8198c1d2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.067048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.068153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.068583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.077034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.078242 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:30.079552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.081694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420c716-1aaf-4462-b20c-7bfc1edfd15f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.088388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] The token cache is being invalidate because project 48cb09ee47f744abae480beb4388fdcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:30.088940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:30.092471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:30.095177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 671/1356] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:29 2022] DELETE /identity/v3/projects/48cb09ee47f744abae480beb4388fdcb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:30.096149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.106952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:30.112246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 686/1357] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:30.112246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.112246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.112246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.171140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:30.172902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:30.182139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 687/1358] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:30.183435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.184909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:30.198589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39984641-e1a4-48c1-a395-9ff18596b624 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.217985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f03ad-e6d3-46ad-a770-3734f0fd068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.221833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f03ad-e6d3-46ad-a770-3734f0fd068c admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.222135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f03ad-e6d3-46ad-a770-3734f0fd068c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.226261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f03ad-e6d3-46ad-a770-3734f0fd068c admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.226261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f03ad-e6d3-46ad-a770-3734f0fd068c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.226261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f03ad-e6d3-46ad-a770-3734f0fd068c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.254884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb139b-6b62-49d1-9662-15846e1c0b74 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.265342 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.268384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.271075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.271075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.273049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.279246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.351191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 688/1359] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:30 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:30.355094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.355564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.356367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.413980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:30.415235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:30.440729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 672/1360] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:30 2022] POST /identity/v3/users => generated 288 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:30.442291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.459605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:30.468435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecfe1e-90e5-4041-9100-e469de83f89e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.542684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 689/1361] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:30.543836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.586374 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2a00-b01f-498c-b8a6-c8cdb14a62ce None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.596594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.597292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.597568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.598147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.602802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.606775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.607583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408fa17-f8f9-4821-bb31-53e0487e7bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.608530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408fa17-f8f9-4821-bb31-53e0487e7bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.608954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408fa17-f8f9-4821-bb31-53e0487e7bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.609413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408fa17-f8f9-4821-bb31-53e0487e7bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.611279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408fa17-f8f9-4821-bb31-53e0487e7bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.613891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408fa17-f8f9-4821-bb31-53e0487e7bc9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.637640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 690/1362] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:30.637640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.637640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.638065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.705275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] MFA Rules not processed for user `a8706e96002946b2b9bfe87de5c3271a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:30.707636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:30.810797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 673/1363] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:30 2022] POST /identity/v3/users => generated 284 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:30.818556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.818687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 691/1364] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:30.830038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.839989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7ae4-980c-426d-8bfb-f30499b0d87f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.857308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fc20a-61a2-49fb-b620-ae05292521d8 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:30.859927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.861610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.862020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.862516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] PATH_INFO: `/v3/projects/23575ee2091540688ca84aa9db0a2c67/users/398dffe8e2e24bdd937ecd947e499505/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.878551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:30.879479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:30.879911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:30.880335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:30.881245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:30.881715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.883552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.908793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23575ee2091540688ca84aa9db0a2c67, user_id=398dffe8e2e24bdd937ecd947e499505, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:30.908793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:30.946947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 674/1365] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:30.948856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:30.991012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ffc4d-7d8a-4f9d-8e31-c1b0fe0d3141 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.010667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bdaaa-9cc0-46a7-aca9-67bd85ddf959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.017372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bdaaa-9cc0-46a7-aca9-67bd85ddf959 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.017372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bdaaa-9cc0-46a7-aca9-67bd85ddf959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.017372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bdaaa-9cc0-46a7-aca9-67bd85ddf959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.017372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bdaaa-9cc0-46a7-aca9-67bd85ddf959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.024289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bdaaa-9cc0-46a7-aca9-67bd85ddf959 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:31.058359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 675/1366] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:31.061138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.062916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.068996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.077660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 692/1367] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:30 2022] PUT /identity/v3/projects/23575ee2091540688ca84aa9db0a2c67/users/398dffe8e2e24bdd937ecd947e499505/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:31.077660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.099885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] MFA Rules not processed for user `a8706e96002946b2b9bfe87de5c3271a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:31.101274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:31.200308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 676/1368] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:31.202009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.311406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9a00-320c-4a78-85fe-c3cb0194c78e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.331990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.332279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.332659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.333150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.334300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.336344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:31.377983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269e9da-5fd0-40ab-a760-885819c00174 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.386124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:31.389718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.390452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.390912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.391354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.392363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.394148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:31.415187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 677/1369] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:31.415884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.431847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e13b1-2357-4ec7-bb42-a2f47f4069fe admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.443794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.443794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.443794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.443794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.444426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.444426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:31.455391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 678/1370] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:31.457059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.462801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:31.491977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 693/1371] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:31 2022] POST /identity/v3/projects => generated 409 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:31.491977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.590884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bc8da-302a-4203-b324-bee0831f7d26 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.597074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.597746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.598024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.598294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] PATH_INFO: `/v3/projects/9c7a553c29a346bbb5b74b9ef9a08ad8/users/e64176ef394f4aa3b4bb5060f307b6ce/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.610344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d99696-cab1-4f74-84a6-fe8fe46960a0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.617653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7a553c29a346bbb5b74b9ef9a08ad8, user_id=e64176ef394f4aa3b4bb5060f307b6ce, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.620492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:31.685106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.685753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.686022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.686290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.769445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 694/1372] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:31 2022] PUT /identity/v3/projects/9c7a553c29a346bbb5b74b9ef9a08ad8/users/e64176ef394f4aa3b4bb5060f307b6ce/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:31.769445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.839844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.841495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:31.858764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 679/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:31.862052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:31.884605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bedb8-d035-46c7-ad5c-b0fdb93e6314 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.950918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4b8e5-ca7f-482f-a6e0-67c5d60bb623 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:31.978526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d77d2a-a4ec-4c17-a3cd-f0f4b51f8187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:31.980366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77d2a-a4ec-4c17-a3cd-f0f4b51f8187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:31.985507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77d2a-a4ec-4c17-a3cd-f0f4b51f8187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:31.987014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77d2a-a4ec-4c17-a3cd-f0f4b51f8187 admin admin] PATH_INFO: `/v3/projects/23575ee2091540688ca84aa9db0a2c67/users/398dffe8e2e24bdd937ecd947e499505/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:31.994246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d77d2a-a4ec-4c17-a3cd-f0f4b51f8187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23575ee2091540688ca84aa9db0a2c67, user_id=398dffe8e2e24bdd937ecd947e499505, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:31.998539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d77d2a-a4ec-4c17-a3cd-f0f4b51f8187 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:32.035139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 695/1374] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:31 2022] PUT /identity/v3/projects/23575ee2091540688ca84aa9db0a2c67/users/398dffe8e2e24bdd937ecd947e499505/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:32.037918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721c46-8f83-45d1-a38e-4ea0f6271f52 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.038293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721c46-8f83-45d1-a38e-4ea0f6271f52 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.045560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721c46-8f83-45d1-a38e-4ea0f6271f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.057238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df83cf-8305-4a92-aeec-ed046be521d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:32.057867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df83cf-8305-4a92-aeec-ed046be521d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.058142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df83cf-8305-4a92-aeec-ed046be521d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.058427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df83cf-8305-4a92-aeec-ed046be521d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.059057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df83cf-8305-4a92-aeec-ed046be521d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:32.064276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df83cf-8305-4a92-aeec-ed046be521d2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:32.112478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:32.117174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f3721c46-8f83-45d1-a38e-4ea0f6271f52 None None] MFA Rules not processed for user `921ba33e2e0740278b7b6fb4af248363`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:32.118393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f3721c46-8f83-45d1-a38e-4ea0f6271f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.285827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 680/1375] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:31 2022] POST /identity/v3/users => generated 305 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:32.288185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.289945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.292671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.305964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 696/1376] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:32.321808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787ae6-8103-4ad8-a902-eb62f248a408 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.321808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787ae6-8103-4ad8-a902-eb62f248a408 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.321808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787ae6-8103-4ad8-a902-eb62f248a408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.367650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] MFA Rules not processed for user `e64176ef394f4aa3b4bb5060f307b6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:32.367650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.396992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-15787ae6-8103-4ad8-a902-eb62f248a408 None None] MFA Rules not processed for user `398dffe8e2e24bdd937ecd947e499505`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:32.399816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-15787ae6-8103-4ad8-a902-eb62f248a408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.780534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 681/1377] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:32.780534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:32.786920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 697/1378] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:32.786920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b11d87-03f0-4bba-8172-582c831c179d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.786920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b11d87-03f0-4bba-8172-582c831c179d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.786920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b11d87-03f0-4bba-8172-582c831c179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.786920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e3b11d87-03f0-4bba-8172-582c831c179d None None] MFA Rules not processed for user `921ba33e2e0740278b7b6fb4af248363`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:32.786920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e3b11d87-03f0-4bba-8172-582c831c179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 698/1379] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:32.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] MFA Rules not processed for user `eb0b112497dc4b64a9ea2f09f808646d`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:32.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.789386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4bb4a-faf1-4211-b320-c9261383c481 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:32.789386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:32.789386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.789386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.789771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.789771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:32.789771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:32.789771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 682/1380] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:32.789771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:32.789771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f5e0c-9973-4329-8df3-74efbb9c3f93 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:32.790250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:32.790250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.790250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.790250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.790606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:32.790606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:32.790993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 699/1381] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:32.790993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:32.790993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.955340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708637ac-1742-4706-a7e5-2e9ccc7fa318 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:32.955340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:32.955340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.955340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.955884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:32.955884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:32.955884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:32.955884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 700/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:32.955884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:32.955884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:32.986099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146298f0-bfe8-4ee4-a4f5-fb9762901766 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:32.991963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85970-5834-4730-bc7e-b83a17bc3996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:32.992766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85970-5834-4730-bc7e-b83a17bc3996 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:32.993029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85970-5834-4730-bc7e-b83a17bc3996 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:32.993318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85970-5834-4730-bc7e-b83a17bc3996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.005982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85970-5834-4730-bc7e-b83a17bc3996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.007885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85970-5834-4730-bc7e-b83a17bc3996 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.052238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 683/1383] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:32 2022] POST /identity/v3/projects => generated 369 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:33.052238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.053500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 701/1384] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:33.053500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d707a-5777-4466-9d99-3c8ff62d06a7 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.053500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d707a-5777-4466-9d99-3c8ff62d06a7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.053500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d707a-5777-4466-9d99-3c8ff62d06a7 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.053500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 702/1385] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 14:39:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:33.053500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d707a-5777-4466-9d99-3c8ff62d06a7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.131954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e53c8-07e0-44f7-acf8-c04dc41c027e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:33.138175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024486f-fd47-487d-b8f8-ba77d2795796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:33.138933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024486f-fd47-487d-b8f8-ba77d2795796 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.139350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024486f-fd47-487d-b8f8-ba77d2795796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.139731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024486f-fd47-487d-b8f8-ba77d2795796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.142234 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024486f-fd47-487d-b8f8-ba77d2795796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.144747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024486f-fd47-487d-b8f8-ba77d2795796 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.183849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 684/1386] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:33.188184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.190550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.193175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.206741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d707a-5777-4466-9d99-3c8ff62d06a7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:33.216711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:33.217662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.219650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.220066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.221791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.228245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.289266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:33.290720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:33.338893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 703/1387] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:33 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:33.338893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.356201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dabe6-0885-4e60-b96a-92d434e2eeca admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:33.362540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12772279-4bca-4677-91f5-52aedf9b3378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:33.363265 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772279-4bca-4677-91f5-52aedf9b3378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.363546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772279-4bca-4677-91f5-52aedf9b3378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.363820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772279-4bca-4677-91f5-52aedf9b3378 admin admin] PATH_INFO: `/v3/projects/28b69f3e74994ad19139a94be8c2af97/users/6779055ccb394822aa6bb12bfbdcb427/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.365045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 685/1388] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:33.366070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.383895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69821-d495-4e4e-b035-f35e9c30ec37 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:33.388950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:33.395120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772279-4bca-4677-91f5-52aedf9b3378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b69f3e74994ad19139a94be8c2af97, user_id=6779055ccb394822aa6bb12bfbdcb427, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.396989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.398036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772279-4bca-4677-91f5-52aedf9b3378 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.405167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.405721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.406854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.408874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.425768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 686/1389] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:33.426784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.428529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:33.444618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f889f1-ad53-41fc-9c41-bc6df428dcd6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:33.449038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:33.449668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.449944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.450212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.458924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.460361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.469619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 687/1390] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:33.470425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.484002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786d66c-04ed-49d3-9b26-34de9826a8d0 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:33.492429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b232b2c-58e3-479e-92f3-479aa716640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:33.492429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232b2c-58e3-479e-92f3-479aa716640c admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.492429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232b2c-58e3-479e-92f3-479aa716640c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.492429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232b2c-58e3-479e-92f3-479aa716640c admin admin] PATH_INFO: `/v3/projects/72f37e999bb246aeb53836c7a8079c58/users/43bbbc1af2fc4c1cadf3ae55c797b894/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.527546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232b2c-58e3-479e-92f3-479aa716640c admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f37e999bb246aeb53836c7a8079c58, user_id=43bbbc1af2fc4c1cadf3ae55c797b894, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:33.532440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232b2c-58e3-479e-92f3-479aa716640c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:33.570067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 704/1391] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:33 2022] PUT /identity/v3/projects/28b69f3e74994ad19139a94be8c2af97/users/6779055ccb394822aa6bb12bfbdcb427/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:33.571802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4dcaa-cc05-4970-be12-e9e15b659e20 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.572872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4dcaa-cc05-4970-be12-e9e15b659e20 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.573386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4dcaa-cc05-4970-be12-e9e15b659e20 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.574840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 705/1392] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 14:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:33.582193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.582786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.583259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.633914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:33.635141 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:33.650405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 688/1393] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:33 2022] PUT /identity/v3/projects/72f37e999bb246aeb53836c7a8079c58/users/43bbbc1af2fc4c1cadf3ae55c797b894/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:33.656858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:33.661991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:33.664162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:33.785073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] MFA Rules not processed for user `6779055ccb394822aa6bb12bfbdcb427`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:33.793819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:33.898157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 706/1394] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:33.899219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:33.959987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 689/1395] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:33.960947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.005636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0496dd0-3ad2-4bd3-a0f4-32fb1a98fc96 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.013522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.014243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.014583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.014919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.017597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.019646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.033120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 707/1396] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:34.034780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.044752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e76d1-ad5d-49cb-9d31-9522d996429e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.051726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.052354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.052638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.052950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.053096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b013e-e286-4c66-8283-3dc0ba9fe274 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.055760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:34.059182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.060047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.062689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.062689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.075239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.076545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.088201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 708/1397] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:34.089785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.106864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f1147-a23c-4047-b111-4a82362bf3f0 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.112353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.113051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.113401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.113859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.114588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.117402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.213874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.215618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.227631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 690/1398] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:34.229761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.256385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f99bb-2cc9-49b1-95da-951e365e44d0 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.270236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.270236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.270236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.270236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] PATH_INFO: `/v3/projects/72f37e999bb246aeb53836c7a8079c58/users/43bbbc1af2fc4c1cadf3ae55c797b894/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.270638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f37e999bb246aeb53836c7a8079c58, user_id=43bbbc1af2fc4c1cadf3ae55c797b894, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.274876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.298742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 691/1399] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:34 2022] PUT /identity/v3/projects/72f37e999bb246aeb53836c7a8079c58/users/43bbbc1af2fc4c1cadf3ae55c797b894/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:34.303891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.328718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 709/1400] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:34 2022] POST /identity/v3/projects => generated 409 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:34.331009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.335179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:34.502935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:34.534113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd8b1b-7c75-4fa7-99a2-b22bba35e3d5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.581922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536adaf-289a-4a62-97a6-d60362f770af admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:34.604808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.605659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.605933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.606202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.613490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:34.630065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.632948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2a8da-2ded-49cd-96ba-fe87b9587f2b service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.646279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 710/1401] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:34.647839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.648181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.648456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.699270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:34.700318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:34.700835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:34.701338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:34.717480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] MFA Rules not processed for user `43bbbc1af2fc4c1cadf3ae55c797b894`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:34.720088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:34.851064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:34.856674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:34.872337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 692/1402] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:34.873739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:34.999164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 711/1403] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:35.020097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:35.089686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb58e-d337-48e7-9cc2-c3731627797d service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:35.103910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:35.104630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.104976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.105362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:35.106126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:35.108257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:35.193640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a287aa0-054b-4f00-8847-ffc07537dd5e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:35.209804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:35.210449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.210728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.211007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:35.228836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:35.230737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:35.244623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 712/1404] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:35.251680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:35.254223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 693/1405] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:34 2022] POST /identity/v3/users => generated 311 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:35.271812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a7d47-b9aa-414f-a3fa-5476631abab7 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:35.287276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:35.287376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.288707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.288989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:35.290863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:35.295516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:35.317662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 713/1406] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:35.321105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:35.354958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a940bf-b877-4a07-8b59-c7693902f5a7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:35.398936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:35.400784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:35.412933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:35.413647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.413925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.414195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] PATH_INFO: `/v3/projects/201fa6233952437ca9a47b03ab33642f/users/5f2cb28f248444cfa190363ef557f55c/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:35.419629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f636-4072-4f33-814e-0ffc9867397a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:35.429663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:35.430495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.430899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.431210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] PATH_INFO: `/v3/users/6fceca70c37a47af88d809d2a1425ca4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:35.432506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fceca70c37a47af88d809d2a1425ca4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:35.435142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:35.489925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:35.508720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201fa6233952437ca9a47b03ab33642f, user_id=5f2cb28f248444cfa190363ef557f55c, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:35.514891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:35.638105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 694/1407] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:35 2022] PUT /identity/v3/projects/201fa6233952437ca9a47b03ab33642f/users/5f2cb28f248444cfa190363ef557f55c/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:35.639231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:35.643816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:35.762296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fceca70c37a47af88d809d2a1425ca4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:35.879286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fceca70c37a47af88d809d2a1425ca4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:35.895160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4d487-1a3b-4a4a-8762-0f1467d6dd75 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:35.909033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:35.910068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.910564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.910936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:35.939645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fceca70c37a47af88d809d2a1425ca4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:35.940425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fceca70c37a47af88d809d2a1425ca4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:35.941364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c86128b-73af-4943-82f3-17abd8ae5c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fceca70c37a47af88d809d2a1425ca4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:35.984185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 714/1408] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:35 2022] DELETE /identity/v3/users/6fceca70c37a47af88d809d2a1425ca4 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:35.989393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a2852-2a74-4204-a760-57e798a9d0d8 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:35.991617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a2852-2a74-4204-a760-57e798a9d0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:35.992040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a2852-2a74-4204-a760-57e798a9d0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:36.047616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-399a2852-2a74-4204-a760-57e798a9d0d8 None None] MFA Rules not processed for user `921ba33e2e0740278b7b6fb4af248363`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:36.052525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-399a2852-2a74-4204-a760-57e798a9d0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:36.095439 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:36.100492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:36.120709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 695/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:36.121618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:36.231150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 715/1410] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:36.231150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:36.231150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:36.231150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:36.328187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dce53a-b68b-4b68-94ee-1b2a7b7e14c3 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:36.363601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:36.368120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:36.602114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:36.603386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:36.603871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:36.604343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:36.605547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:36.608554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:36.654800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 716/1411] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:36.662040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:36.662954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 696/1412] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:36.664862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:36.679287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:36.681943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:36.698219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a1f9-66e1-4137-b897-9df94b5e1f81 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:36.706487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024a21d-67fa-4e86-ab6d-917518cf445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:36.707646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a21d-67fa-4e86-ab6d-917518cf445e admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:36.708134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a21d-67fa-4e86-ab6d-917518cf445e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:36.708613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a21d-67fa-4e86-ab6d-917518cf445e admin admin] PATH_INFO: `/v3/projects/201fa6233952437ca9a47b03ab33642f/users/5f2cb28f248444cfa190363ef557f55c/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:36.712668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024a21d-67fa-4e86-ab6d-917518cf445e admin admin] RBAC: Authorizing `identity:create_grant(project_id=201fa6233952437ca9a47b03ab33642f, user_id=5f2cb28f248444cfa190363ef557f55c, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:36.717102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024a21d-67fa-4e86-ab6d-917518cf445e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:36.757723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 717/1413] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:36 2022] PUT /identity/v3/projects/201fa6233952437ca9a47b03ab33642f/users/5f2cb28f248444cfa190363ef557f55c/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:36.768655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:36.769140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:36.769592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:36.863211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] MFA Rules not processed for user `5f2cb28f248444cfa190363ef557f55c`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:36.864724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f27b3b-d656-4522-811b-a99f45b4d43a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:36.867003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:37.031819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:37.032506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:37.034383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:37.034383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:37.196432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 718/1414] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:37.196960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:37.196960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:37.196960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 697/1415] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:39:37.323452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:37.965520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:37.967962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:38.059552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:38.062389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:38.118557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96247548-0113-4984-9420-364e4c4fce15 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:38.140971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:38.144583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:38.144872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:38.145572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] PATH_INFO: `/v3/projects/42718454692446bbb6605fc53d3bae28` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:38.148253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42718454692446bbb6605fc53d3bae28)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:38.152944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:38.168228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d45bab-9a22-4c96-bdba-a50eb212c05b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:38.173598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21e7eb-8f14-426a-ab51-cb0b86e2a62c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:38.174286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21e7eb-8f14-426a-ab51-cb0b86e2a62c service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:38.174589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21e7eb-8f14-426a-ab51-cb0b86e2a62c service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:38.176036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21e7eb-8f14-426a-ab51-cb0b86e2a62c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:38.189616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21e7eb-8f14-426a-ab51-cb0b86e2a62c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:38.192343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21e7eb-8f14-426a-ab51-cb0b86e2a62c service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:38.256986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 719/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:37 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:38.389433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42718454692446bbb6605fc53d3bae28', 'request_id': 'req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:38.413178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42718454692446bbb6605fc53d3bae28', 'request_id': 'req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:38.432065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42718454692446bbb6605fc53d3bae28', 'request_id': 'req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:38.559728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] The token cache is being invalidate because project 42718454692446bbb6605fc53d3bae28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:38.560051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:38.560827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3fc1721c-d1da-45de-b6bb-a06fad99d8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:38.563525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 698/1417] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:38 2022] DELETE /identity/v3/projects/42718454692446bbb6605fc53d3bae28 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:38.568192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:38.568484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:38.568896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:38.620959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:38.622367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:38.785191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 720/1418] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:38.791027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:38.791554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:38.791554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:38.833353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:38.834919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:38.856544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 699/1419] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:38.858906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:38.874640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf596d99-7b7c-4088-9341-ef9a616c32b7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:38.888359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:38.889279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:38.889559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:38.889830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:38.890695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:38.892360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:38.896644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:38.930144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 721/1420] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:38.934383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:38.954319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182cc61-2ab6-4bff-80fe-db265a126520 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:38.972895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:38.975575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:38.979683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:38.979683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:38.979683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:38.988006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:39.095296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 700/1421] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:38 2022] POST /identity/v3/projects => generated 355 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:39.098325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:39.163789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:39.165934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:39.176597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81223c5d-c8a6-4324-b049-5fc9a18cd6e1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:39.288774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:39.289779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:39.290253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:39.291274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:39.292113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:39.303123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:39.365219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25f27-6e3b-4ce4-981e-ace3860b2237 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:39.369766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:39.370835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:39.371292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:39.371683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:39.374375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:39.470407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:39.472518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:39.476225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 722/1422] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:39 2022] POST /identity/v3/users => generated 278 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:39.480899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:39.498231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75418186-0863-4048-81b5-bce6f9f0b6ca admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:39.498435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:39.500745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:39.501073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:39.501554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:39.502077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:39.503716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:39.506870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 701/1423] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:39.552484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 723/1424] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:39.559860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:39.577402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cf8da-0768-4a20-9905-45b760bf25b8 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:39.606694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77ece2-8c76-48ec-8845-6c4f79c44e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:39.607404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77ece2-8c76-48ec-8845-6c4f79c44e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:39.607686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77ece2-8c76-48ec-8845-6c4f79c44e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:39.608390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77ece2-8c76-48ec-8845-6c4f79c44e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:39.608610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77ece2-8c76-48ec-8845-6c4f79c44e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:39.610608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77ece2-8c76-48ec-8845-6c4f79c44e1f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:39.634069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 702/1425] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:39.634161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:39.653146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6614-2759-41cf-817a-1612930ee103 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:39.658502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:39.659971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:39.670617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:39.670617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] PATH_INFO: `/v3/projects/ad47caf1520a49dc83d4bc12455a854e/users/7fd21cedfb844c3893b936ab6cf5b047/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:39.689272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad47caf1520a49dc83d4bc12455a854e, user_id=7fd21cedfb844c3893b936ab6cf5b047, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:39.691918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:39.799307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 724/1426] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:39 2022] PUT /identity/v3/projects/ad47caf1520a49dc83d4bc12455a854e/users/7fd21cedfb844c3893b936ab6cf5b047/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:39.811061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:39.811412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:39.811801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:39.841888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] MFA Rules not processed for user `7fd21cedfb844c3893b936ab6cf5b047`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:39.843094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:39.917191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 703/1427] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:39.922761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:40.008933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069315b1-9e73-4976-bfbc-aeec97f9e1e6 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:40.013769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a03ac-420d-4413-84c8-1363a6c3e4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:40.014376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a03ac-420d-4413-84c8-1363a6c3e4da admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:40.014653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a03ac-420d-4413-84c8-1363a6c3e4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:40.014924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a03ac-420d-4413-84c8-1363a6c3e4da admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:40.015611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a03ac-420d-4413-84c8-1363a6c3e4da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:40.017679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a03ac-420d-4413-84c8-1363a6c3e4da admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:40.098161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 725/1428] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:39 2022] POST /identity/v3/projects => generated 353 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:39:40.102724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:40.115060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3559586-2558-4898-ab23-c929d9995a2e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:40.224225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:40.225480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:40.226019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:40.226496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:40.227436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:40.230532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:40.244873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:40.347032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:40.363936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:40.364221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:40.364741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:40.390970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 704/1429] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:40 2022] POST /identity/v3/users => generated 283 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:40.397327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:40.402103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] MFA Rules not processed for user `eb0b112497dc4b64a9ea2f09f808646d`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:40.405175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:40.418123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41982d-9d48-4d33-b080-4a78b170ab2e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:40.430451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:40.431096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:40.431420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:40.431636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:40.432451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:40.434299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:40.480154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 705/1430] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:40.481115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:40.507062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a833-8cf8-4f02-9089-bcf62ba850a3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:40.513975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:40.514598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:40.514882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:40.515159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] PATH_INFO: `/v3/projects/8e051c99570e4bb1bb57e186ee5f566c/users/d4051ad4e8bf44dd9ea868a8bfdc9267/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:40.544529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 726/1431] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:40.566684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e051c99570e4bb1bb57e186ee5f566c, user_id=d4051ad4e8bf44dd9ea868a8bfdc9267, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:40.570499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:40.785162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 706/1432] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:40 2022] PUT /identity/v3/projects/8e051c99570e4bb1bb57e186ee5f566c/users/d4051ad4e8bf44dd9ea868a8bfdc9267/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:40.789398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:40.809201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f86f76-0c57-4e65-99bd-9d2859b40f65 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:40.951321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08b58-c0c6-4c1a-8016-4ff56ee7e891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:40.952022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b58-c0c6-4c1a-8016-4ff56ee7e891 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:40.952422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b58-c0c6-4c1a-8016-4ff56ee7e891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:40.952702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b58-c0c6-4c1a-8016-4ff56ee7e891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:40.954246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f08b58-c0c6-4c1a-8016-4ff56ee7e891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:40.956363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f08b58-c0c6-4c1a-8016-4ff56ee7e891 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:40.971517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 727/1433] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:40.975128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:41.026268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbf6f-c39e-406e-b3fb-23e5184e9158 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:41.042853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:41.043682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:41.043972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:41.044242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] PATH_INFO: `/v3/projects/8e051c99570e4bb1bb57e186ee5f566c/users/d4051ad4e8bf44dd9ea868a8bfdc9267/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:41.049956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e051c99570e4bb1bb57e186ee5f566c, user_id=d4051ad4e8bf44dd9ea868a8bfdc9267, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:41.054390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:41.066511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 707/1434] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:40 2022] PUT /identity/v3/projects/8e051c99570e4bb1bb57e186ee5f566c/users/d4051ad4e8bf44dd9ea868a8bfdc9267/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:41.076888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3715af2-0ac1-471d-a48c-e76c7a48714e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:41.077643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3715af2-0ac1-471d-a48c-e76c7a48714e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:41.079840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3715af2-0ac1-471d-a48c-e76c7a48714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:41.123005 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f3715af2-0ac1-471d-a48c-e76c7a48714e None None] MFA Rules not processed for user `d4051ad4e8bf44dd9ea868a8bfdc9267`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:41.124324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f3715af2-0ac1-471d-a48c-e76c7a48714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:41.242998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 728/1435] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:41.306188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:41.388700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1424cb1-be3d-477a-a48f-62d04387c747 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:41.393993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:41.394901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:41.395446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:41.395998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:41.407224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:41.409130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:41.425676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 708/1436] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:41.526014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:41.526014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:41.526014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:41.583650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:41.585109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:41.745219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 729/1437] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:41.962895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:41.977811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753a3c9-ea38-4fb3-8152-61b95f0e02d9 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:41.983340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:41.987224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:41.987224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:41.987224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:42.100939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:42.101907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:42.184669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:42.186448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:42.212127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 709/1438] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:42.311223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15510fe2-dfaf-4acc-93dd-21fae4c48948 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:42.335955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3dca-6e93-4c12-adf3-1d84a139ab5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:42.338933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3dca-6e93-4c12-adf3-1d84a139ab5f service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:42.339403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3dca-6e93-4c12-adf3-1d84a139ab5f service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:42.340562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3dca-6e93-4c12-adf3-1d84a139ab5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:42.373660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d3dca-6e93-4c12-adf3-1d84a139ab5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:42.373660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d3dca-6e93-4c12-adf3-1d84a139ab5f service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:42.381414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 730/1439] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:39:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:39:43.610181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:43.715514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:44.474430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:44.475879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:44.540363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa8d0e-f0ff-4151-aacb-e6bfc0ed8ebf service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:44.544861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:44.545518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:44.545792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:44.546060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] PATH_INFO: `/v3/users/e64176ef394f4aa3b4bb5060f307b6ce` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:44.547840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e64176ef394f4aa3b4bb5060f307b6ce)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:44.549814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:44.626462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:44.759664 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e64176ef394f4aa3b4bb5060f307b6ce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:44.771722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e64176ef394f4aa3b4bb5060f307b6ce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:44.848977 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e64176ef394f4aa3b4bb5060f307b6ce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:44.849703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e64176ef394f4aa3b4bb5060f307b6ce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:44.850308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e64176ef394f4aa3b4bb5060f307b6ce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:44.870421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 710/1440] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:44 2022] DELETE /identity/v3/users/e64176ef394f4aa3b4bb5060f307b6ce => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:44.875023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:44.875426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:44.875634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:44.895160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:44.896359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:44.955410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 731/1441] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:44.965511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:44.966992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:45.072414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25e676-3478-4848-b529-e7eda198ea81 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:45.076390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:45.077027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:45.077334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:45.078309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:45.096017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:45.097594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:45.109249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 711/1442] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:45.875694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:45.877241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:45.909651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b228b14-96f8-42db-8c84-f89ea403b740 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:45.915877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:45.916781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:45.917656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:45.918068 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] PATH_INFO: `/v3/projects/9c7a553c29a346bbb5b74b9ef9a08ad8` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:45.919838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c7a553c29a346bbb5b74b9ef9a08ad8)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:45.926046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:46.146345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:46.220525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c7a553c29a346bbb5b74b9ef9a08ad8', 'request_id': 'req-0270d492-2f44-4504-966f-c0ed6081b4dc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.227103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c7a553c29a346bbb5b74b9ef9a08ad8', 'request_id': 'req-0270d492-2f44-4504-966f-c0ed6081b4dc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.345668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c7a553c29a346bbb5b74b9ef9a08ad8', 'request_id': 'req-0270d492-2f44-4504-966f-c0ed6081b4dc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.368692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] The token cache is being invalidate because project 9c7a553c29a346bbb5b74b9ef9a08ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:46.368692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.368692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.368692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 732/1443] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:45 2022] DELETE /identity/v3/projects/9c7a553c29a346bbb5b74b9ef9a08ad8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:46.371078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:46.381789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf42cbb-857d-43e6-af7c-3d181c6816b6 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:46.504325 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:46.505159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:46.505443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:46.505724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] PATH_INFO: `/v3/users/43bbbc1af2fc4c1cadf3ae55c797b894` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:46.507540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] RBAC: Authorizing `identity:delete_user(user_id=43bbbc1af2fc4c1cadf3ae55c797b894)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:46.509178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:46.791198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43bbbc1af2fc4c1cadf3ae55c797b894'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.805472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43bbbc1af2fc4c1cadf3ae55c797b894'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.882828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43bbbc1af2fc4c1cadf3ae55c797b894'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.883572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43bbbc1af2fc4c1cadf3ae55c797b894'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.884194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44658943-36fa-4f44-85d8-22fe2460e0be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43bbbc1af2fc4c1cadf3ae55c797b894'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:46.898351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 712/1444] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:46 2022] DELETE /identity/v3/users/43bbbc1af2fc4c1cadf3ae55c797b894 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:47.256469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:47.366287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:47.369158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:47.384516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d492-2f44-4504-966f-c0ed6081b4dc admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:47.400010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:47.457835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:47.458500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:47.458837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:47.459131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] PATH_INFO: `/v3/projects/72f37e999bb246aeb53836c7a8079c58` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:47.460227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f37e999bb246aeb53836c7a8079c58)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:47.462408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:47.604152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f37e999bb246aeb53836c7a8079c58', 'request_id': 'req-0ca79074-27ca-4c0e-b52d-119aa70c897c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:47.616076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f37e999bb246aeb53836c7a8079c58', 'request_id': 'req-0ca79074-27ca-4c0e-b52d-119aa70c897c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:47.728295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f37e999bb246aeb53836c7a8079c58', 'request_id': 'req-0ca79074-27ca-4c0e-b52d-119aa70c897c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:47.740653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] The token cache is being invalidate because project 72f37e999bb246aeb53836c7a8079c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:47.741017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:47.741709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0ca79074-27ca-4c0e-b52d-119aa70c897c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:47.743519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 733/1445] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:47 2022] DELETE /identity/v3/projects/72f37e999bb246aeb53836c7a8079c58 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:47.748439 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:47.749018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:47.749135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:47.774974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:47.776177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:47.838316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 713/1446] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:47.846839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:47.847240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:47.847586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:47.870434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:47.871649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:47.882171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 734/1447] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:47.886568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:47.905882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d0d8b-5928-4085-be99-14d413ead2c8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:47.911797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:47.912408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:47.912679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:47.912942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:47.915221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:39:47.915221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:47.916211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:47.943986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 714/1448] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:39:47.948783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:47.958387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b850c2f-d6ac-47b2-9ec6-a7c47f4b79ff None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:47.977114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:47.977114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:47.979803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:47.980179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:47.980980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:47.983048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:48.094388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 735/1449] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:47 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:48.095713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:48.106966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3952e1c-78af-4bb8-a2dd-7f0814a3050c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:48.184638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:48.185468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:48.185750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:48.186248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:48.187457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:48.190216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:48.324328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 715/1450] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:48 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:48.334333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:48.346398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b874cfe-98d2-492e-bba6-17a0020f85aa admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:48.357703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:48.363678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:48.397651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:48.397651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:48.397651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:48.397651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:48.397651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 736/1451] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:48.401237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:48.415840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c69c03-5911-462d-bf4e-fd3fc5f8a1b4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:48.443403 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47f738-92e0-49a6-9913-28d683733c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:48.446665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47f738-92e0-49a6-9913-28d683733c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:48.446665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47f738-92e0-49a6-9913-28d683733c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:48.446665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47f738-92e0-49a6-9913-28d683733c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:48.446665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47f738-92e0-49a6-9913-28d683733c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:48.448004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47f738-92e0-49a6-9913-28d683733c28 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:48.461307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 716/1452] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:48.465366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:48.484041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b997-b83e-49fd-9ac3-8c40c753f757 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:48.498082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:48.499016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:48.499626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:48.499977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] PATH_INFO: `/v3/projects/7d2acf17fac34805a9e19d067e4adbc7/users/31fc9a5e50c246bd8fc06ebc7d708591/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:48.519429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2acf17fac34805a9e19d067e4adbc7, user_id=31fc9a5e50c246bd8fc06ebc7d708591, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:48.524510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:49.612054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 737/1453] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:48 2022] PUT /identity/v3/projects/7d2acf17fac34805a9e19d067e4adbc7/users/31fc9a5e50c246bd8fc06ebc7d708591/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 1144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:49.622221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:49.623848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:49.625266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:49.661245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] MFA Rules not processed for user `31fc9a5e50c246bd8fc06ebc7d708591`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:49.662572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:49.732916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 717/1454] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:49.738697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:49.822974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c815a-4e24-4490-acfe-9ff44290bc00 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:49.827646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:49.828294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:49.828575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:49.828841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:49.829530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:49.831441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:50.034310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 738/1455] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:39:49 2022] POST /identity/v3/projects => generated 379 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:50.038749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:50.050144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6e23-68ec-436b-a94c-2244a7ae807f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:50.182467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:50.217556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:50.217556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:50.217556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:50.217556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:50.217951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:50.217951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:50.288987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:50.363523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 718/1456] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:39:50 2022] POST /identity/v3/users => generated 296 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:39:50.364818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:50.385028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800148d2-29e7-4afd-9a58-0bb01e96a9cb admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:50.411053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:50.412130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:50.414957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:50.415334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:50.417970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:50.420120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:50.459194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 739/1457] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:50.464179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:50.479109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0710a-fe64-4c2b-a1b9-e98d4925d7aa admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:50.500537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:50.501300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:50.501534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:50.501808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] PATH_INFO: `/v3/projects/40c635b4a5054b959f2a2b33eb3b56ba/users/5057d2b923934f4d8133c4d61d510249/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:50.534235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c635b4a5054b959f2a2b33eb3b56ba, user_id=5057d2b923934f4d8133c4d61d510249, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:50.536865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:50.714881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 719/1458] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:50 2022] PUT /identity/v3/projects/40c635b4a5054b959f2a2b33eb3b56ba/users/5057d2b923934f4d8133c4d61d510249/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:50.719891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:50.736689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad6686-bbae-4451-8298-a88babf771e3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:50.819045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ada2a-7740-4e98-8389-445739424639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:50.820128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ada2a-7740-4e98-8389-445739424639 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:50.820649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ada2a-7740-4e98-8389-445739424639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:50.821132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ada2a-7740-4e98-8389-445739424639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:50.827885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ada2a-7740-4e98-8389-445739424639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:50.827885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ada2a-7740-4e98-8389-445739424639 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:50.834498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 740/1459] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:39:50.839312 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:50.854827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452528e-37a0-47e3-af1e-c22e0e187b05 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:50.863369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:50.864334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:50.866057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:50.866417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] PATH_INFO: `/v3/projects/40c635b4a5054b959f2a2b33eb3b56ba/users/5057d2b923934f4d8133c4d61d510249/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:50.869767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c635b4a5054b959f2a2b33eb3b56ba, user_id=5057d2b923934f4d8133c4d61d510249, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:50.873728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:50.896608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 720/1460] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:39:50 2022] PUT /identity/v3/projects/40c635b4a5054b959f2a2b33eb3b56ba/users/5057d2b923934f4d8133c4d61d510249/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:50.907165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:50.907470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:50.907794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:50.954773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] MFA Rules not processed for user `5057d2b923934f4d8133c4d61d510249`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:50.956137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:51.069340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 741/1461] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:51.160601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:51.162402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:51.236516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401ce9c-ecfc-49c1-ac0d-84f9dbd854b7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:51.240109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccc1a0-db2d-4970-a1ee-78c47e4987b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:51.240857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc1a0-db2d-4970-a1ee-78c47e4987b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:51.241245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc1a0-db2d-4970-a1ee-78c47e4987b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:51.241515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc1a0-db2d-4970-a1ee-78c47e4987b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:51.256375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc1a0-db2d-4970-a1ee-78c47e4987b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:51.258065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc1a0-db2d-4970-a1ee-78c47e4987b4 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:51.274651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 721/1462] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:51.466594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:51.478950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24092f4b-39a0-4c67-854b-4d73a3ae8749 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:51.484030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:51.484836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:51.485218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:51.485601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:51.562308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:51.574071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:51.580713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 742/1463] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:53.162828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:53.358169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:53.894896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:53.895457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:53.895832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:53.937778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:53.942174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:54.017317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 722/1464] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Nov 2 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:54.136749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:54.138866 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:54.163975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be17b3-5cbd-4902-bee7-26d0ce3acb4c service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:54.176465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:54.177321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:54.178837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:54.179259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:54.190594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:54.195938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:54.206949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 743/1465] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:54.219677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:54.221057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:54.237203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e9936-2834-4b62-ad2e-cdc22e1fcccd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:54.250857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc2f83-3c49-479d-97e7-1c2d13815e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:54.251768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2f83-3c49-479d-97e7-1c2d13815e65 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:54.252210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2f83-3c49-479d-97e7-1c2d13815e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:54.252542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2f83-3c49-479d-97e7-1c2d13815e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:54.267427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc2f83-3c49-479d-97e7-1c2d13815e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:54.272010 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc2f83-3c49-479d-97e7-1c2d13815e65 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:54.291612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 723/1466] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:55.408347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:55.410213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:55.573609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35821f2-0819-43fb-a574-66d6e969feb1 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:55.589548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:55.589548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:55.589548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:55.589548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] PATH_INFO: `/v3/users/6779055ccb394822aa6bb12bfbdcb427` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:55.595396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6779055ccb394822aa6bb12bfbdcb427)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:55.615626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:55.892909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08866e6c-4a88-48e5-972a-efc08883f01e None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:55.893855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08866e6c-4a88-48e5-972a-efc08883f01e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:55.894467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08866e6c-4a88-48e5-972a-efc08883f01e None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:55.896951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 724/1467] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:39:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:55.911108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6779055ccb394822aa6bb12bfbdcb427'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:55.920300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:55.921853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:55.922460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:55.956104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:55.958071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:55.984895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6779055ccb394822aa6bb12bfbdcb427'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.011718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6779055ccb394822aa6bb12bfbdcb427'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.013396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6779055ccb394822aa6bb12bfbdcb427'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.014933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-86de8692-0f3c-4f0b-a2a2-a22c1f3b0985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6779055ccb394822aa6bb12bfbdcb427'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.041440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 744/1468] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:55 2022] DELETE /identity/v3/users/6779055ccb394822aa6bb12bfbdcb427 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:56.044249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.044751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.045372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.078487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:56.080525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:56.192089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 725/1469] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:39:56.193655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:56.196937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:56.224042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd091cc-2f55-4861-b64b-1650efba4ca6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:56.280651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 745/1470] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:56.282357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:56.283855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:56.309704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:56.378055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:56.379040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.379659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.380173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] PATH_INFO: `/v3/users/0c1c8080db1249228e6ec4b5a690a9e5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.381586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1c8080db1249228e6ec4b5a690a9e5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:56.384432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:56.430000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1cb6-7202-477c-99f4-0adec865befc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:56.444830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:56.446537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.447674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.448111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.467186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:56.470528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:56.492578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 746/1471] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:56.494529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:56.500734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:56.650042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27699b8-73bd-4af2-adf9-618992859e05 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:56.654834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:56.655966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.656249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.656518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.667337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:56.668932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:56.681491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 747/1472] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:56.682315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:56.683546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:56.696917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db290f83-1bac-4008-8616-f263429e2b15 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:56.705679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:56.706989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.706989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.706989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] PATH_INFO: `/v3/users/31fc9a5e50c246bd8fc06ebc7d708591` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.708132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fc9a5e50c246bd8fc06ebc7d708591)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:56.710145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:56.721688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1c8080db1249228e6ec4b5a690a9e5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.740080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1c8080db1249228e6ec4b5a690a9e5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.900184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1c8080db1249228e6ec4b5a690a9e5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.901429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1c8080db1249228e6ec4b5a690a9e5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.902930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ec9c40d-7a88-4d05-8cac-ad15f11fac64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1c8080db1249228e6ec4b5a690a9e5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:56.920647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 726/1473] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:56 2022] DELETE /identity/v3/users/0c1c8080db1249228e6ec4b5a690a9e5 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:56.922363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b8eb-4ee7-4949-90b7-0f03f5585fe0 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.922863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b8eb-4ee7-4949-90b7-0f03f5585fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.923366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b8eb-4ee7-4949-90b7-0f03f5585fe0 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.925194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 727/1474] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:39:56.926448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b985-c80a-4873-87a0-b3d3a43783dd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:56.926880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b985-c80a-4873-87a0-b3d3a43783dd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:56.927151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b985-c80a-4873-87a0-b3d3a43783dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:56.954201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1370b985-c80a-4873-87a0-b3d3a43783dd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:56.956805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1370b985-c80a-4873-87a0-b3d3a43783dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:57.036117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fc9a5e50c246bd8fc06ebc7d708591'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:57.071472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 728/1475] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:57.073157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2768aa-3a1d-4d73-802c-f741223a4e5e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:57.073624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2768aa-3a1d-4d73-802c-f741223a4e5e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:57.074314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2768aa-3a1d-4d73-802c-f741223a4e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:57.095942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fc9a5e50c246bd8fc06ebc7d708591'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:57.104314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6a2768aa-3a1d-4d73-802c-f741223a4e5e None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:57.105827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6a2768aa-3a1d-4d73-802c-f741223a4e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:57.139845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fc9a5e50c246bd8fc06ebc7d708591'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:57.142814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fc9a5e50c246bd8fc06ebc7d708591'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:57.146268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fc9a5e50c246bd8fc06ebc7d708591'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:57.203938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 748/1476] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:56 2022] DELETE /identity/v3/users/31fc9a5e50c246bd8fc06ebc7d708591 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:57.204017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:57.206309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:57.254561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 729/1477] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:39:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:57.255767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:57.256046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:57.256317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:57.279797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:39:57.281687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:57.362584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a99d2-4929-4f90-972c-4939f8312732 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:57.374384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:57.375026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:57.375301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:57.375565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:57.382616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:57.416366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 730/1478] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:39:57.440951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:57.440951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:57.447329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:57.450785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:57.466456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 749/1479] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:57.467802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:57.477434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48c25e-d067-4cfa-9669-50c554e2b644 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:57.481867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:57.486619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:57.486619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:57.486619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:57.497515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:57.499134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:57.511343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 750/1480] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:39:57.513632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:57.551203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1d67-7b1f-4b12-be68-7c34c7aaae25 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:57.557519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:57.558646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:57.558646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:57.558869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:57.570460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:57.573060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:57.594337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 731/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:39:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:39:58.014857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:58.491735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:58.493305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:58.521983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf38c68-064b-496d-807e-3f5f03f61252 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:58.549932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:58.549932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:58.549932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:58.549932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] PATH_INFO: `/v3/projects/28b69f3e74994ad19139a94be8c2af97` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:58.556161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b69f3e74994ad19139a94be8c2af97)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:58.558838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:59.173765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:59.174617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:59.197803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b69f3e74994ad19139a94be8c2af97', 'request_id': 'req-425c45a2-569d-4702-bf7b-a5ca5d6c2093'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.204812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b22d0e-29ed-40c9-a1d0-2d375e3587ca service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:59.206253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b69f3e74994ad19139a94be8c2af97', 'request_id': 'req-425c45a2-569d-4702-bf7b-a5ca5d6c2093'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.321261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:59.322092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:59.322371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:59.322668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] PATH_INFO: `/v3/projects/7d2acf17fac34805a9e19d067e4adbc7` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:59.323895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d2acf17fac34805a9e19d067e4adbc7)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:59.328887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:59.345290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b69f3e74994ad19139a94be8c2af97', 'request_id': 'req-425c45a2-569d-4702-bf7b-a5ca5d6c2093'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.362324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] The token cache is being invalidate because project 28b69f3e74994ad19139a94be8c2af97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:59.362706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.365433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.368708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 751/1482] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:58 2022] DELETE /identity/v3/projects/28b69f3e74994ad19139a94be8c2af97 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:59.369978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:59.371422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:59.388514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c45a2-569d-4702-bf7b-a5ca5d6c2093 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:59.396388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:59.397798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:59.398363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:59.398802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] PATH_INFO: `/v3/projects/5652c00304d64379a5802b52c9f298f5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:59.400032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5652c00304d64379a5802b52c9f298f5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:59.402413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:59.528560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d2acf17fac34805a9e19d067e4adbc7', 'request_id': 'req-158a95d8-ac7d-4186-912c-5472ed0cdcee'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.536449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d2acf17fac34805a9e19d067e4adbc7', 'request_id': 'req-158a95d8-ac7d-4186-912c-5472ed0cdcee'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.543015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d2acf17fac34805a9e19d067e4adbc7', 'request_id': 'req-158a95d8-ac7d-4186-912c-5472ed0cdcee'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.551423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5652c00304d64379a5802b52c9f298f5', 'request_id': 'req-30eae3ca-b9da-4224-a763-7eeec2a4d360'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.558535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5652c00304d64379a5802b52c9f298f5', 'request_id': 'req-30eae3ca-b9da-4224-a763-7eeec2a4d360'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.590172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:39:59.660296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] The token cache is being invalidate because project 7d2acf17fac34805a9e19d067e4adbc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:59.660631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.661379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.662980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 732/1483] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:59 2022] DELETE /identity/v3/projects/7d2acf17fac34805a9e19d067e4adbc7 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:59.663826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:59.665170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:59.736858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a95d8-ac7d-4186-912c-5472ed0cdcee admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:59.750974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5652c00304d64379a5802b52c9f298f5', 'request_id': 'req-30eae3ca-b9da-4224-a763-7eeec2a4d360'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.775426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] The token cache is being invalidate because project 5652c00304d64379a5802b52c9f298f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:39:59.776264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.777439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:39:59.779248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 752/1484] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:39:59 2022] DELETE /identity/v3/projects/5652c00304d64379a5802b52c9f298f5 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:39:59.780904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:39:59.782719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:39:59.802586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:59.803420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:59.803697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:59.804157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] PATH_INFO: `/v3/users/5f2cb28f248444cfa190363ef557f55c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:59.805281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2cb28f248444cfa190363ef557f55c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:59.807268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:39:59.856715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eae3ca-b9da-4224-a763-7eeec2a4d360 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:39:59.865148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:39:59.865148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:39:59.865148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:39:59.865148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] PATH_INFO: `/v3/users/5057d2b923934f4d8133c4d61d510249` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:39:59.866575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5057d2b923934f4d8133c4d61d510249)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:39:59.868635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:00.056419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2cb28f248444cfa190363ef557f55c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.069516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2cb28f248444cfa190363ef557f55c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.178636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5057d2b923934f4d8133c4d61d510249'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.200120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2cb28f248444cfa190363ef557f55c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.200855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2cb28f248444cfa190363ef557f55c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.204688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2cb28f248444cfa190363ef557f55c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.226096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 733/1485] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:59 2022] DELETE /identity/v3/users/5f2cb28f248444cfa190363ef557f55c => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:00.226096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:00.226096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:00.323643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83593441-e6c0-4629-acaf-aa5c166c5e1e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:00.332434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:00.334145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:00.334549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:00.334916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] PATH_INFO: `/v3/users/398dffe8e2e24bdd937ecd947e499505` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:00.336659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=398dffe8e2e24bdd937ecd947e499505)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:00.339492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:00.346077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5057d2b923934f4d8133c4d61d510249'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.373129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5057d2b923934f4d8133c4d61d510249'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.374218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5057d2b923934f4d8133c4d61d510249'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.374959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5057d2b923934f4d8133c4d61d510249'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.402578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 753/1486] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:39:59 2022] DELETE /identity/v3/users/5057d2b923934f4d8133c4d61d510249 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:00.643553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398dffe8e2e24bdd937ecd947e499505'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.672621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398dffe8e2e24bdd937ecd947e499505'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.797267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398dffe8e2e24bdd937ecd947e499505'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.797993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398dffe8e2e24bdd937ecd947e499505'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.798696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398dffe8e2e24bdd937ecd947e499505'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:00.813834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 734/1487] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:00 2022] DELETE /identity/v3/users/398dffe8e2e24bdd937ecd947e499505 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:01.547639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:01.570476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:01.572038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:01.672390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fac078-45b1-4758-9a63-224051767ca2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:01.683282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:01.740109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:01.740744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:01.743143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:01.743505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] PATH_INFO: `/v3/projects/23575ee2091540688ca84aa9db0a2c67` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:01.744811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] RBAC: Authorizing `identity:delete_project(project_id=23575ee2091540688ca84aa9db0a2c67)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:01.746841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:01.772791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:01.775460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:01.812788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d0a30-82d1-4bd0-9944-b7423241b42d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:01.820292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:01.821424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:01.821933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:01.822988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] PATH_INFO: `/v3/projects/201fa6233952437ca9a47b03ab33642f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:01.824570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=201fa6233952437ca9a47b03ab33642f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:01.827957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:01.949162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201fa6233952437ca9a47b03ab33642f', 'request_id': 'req-2ef7a868-8666-48ef-9df3-f231b5b8abeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:01.957215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201fa6233952437ca9a47b03ab33642f', 'request_id': 'req-2ef7a868-8666-48ef-9df3-f231b5b8abeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:01.963100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201fa6233952437ca9a47b03ab33642f', 'request_id': 'req-2ef7a868-8666-48ef-9df3-f231b5b8abeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.052375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23575ee2091540688ca84aa9db0a2c67', 'request_id': 'req-b0634d24-a0aa-43ea-a3a7-a8431db7969d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.053926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] The token cache is being invalidate because project 201fa6233952437ca9a47b03ab33642f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:02.054220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.059250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.062459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 735/1488] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:01 2022] DELETE /identity/v3/projects/201fa6233952437ca9a47b03ab33642f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:02.063464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.067285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23575ee2091540688ca84aa9db0a2c67', 'request_id': 'req-b0634d24-a0aa-43ea-a3a7-a8431db7969d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.068436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.160156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7a868-8666-48ef-9df3-f231b5b8abeb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.167945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23575ee2091540688ca84aa9db0a2c67', 'request_id': 'req-b0634d24-a0aa-43ea-a3a7-a8431db7969d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.192400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] The token cache is being invalidate because project 23575ee2091540688ca84aa9db0a2c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:02.192855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.196121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b0634d24-a0aa-43ea-a3a7-a8431db7969d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.201916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 754/1489] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:01 2022] DELETE /identity/v3/projects/23575ee2091540688ca84aa9db0a2c67 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:02.201916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc641-d943-4fce-9d85-749a74fba770 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.204881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc641-d943-4fce-9d85-749a74fba770 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.204881 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc641-d943-4fce-9d85-749a74fba770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.266915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-925fc641-d943-4fce-9d85-749a74fba770 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:02.268187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-925fc641-d943-4fce-9d85-749a74fba770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.273879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.275023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.275493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.276050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] PATH_INFO: `/v3/projects/40c635b4a5054b959f2a2b33eb3b56ba` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.277897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c635b4a5054b959f2a2b33eb3b56ba)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.281868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:02.288015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 755/1490] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:02.289442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038e95-0c5c-4066-a77b-2f71810c0e11 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.294105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038e95-0c5c-4066-a77b-2f71810c0e11 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.294105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038e95-0c5c-4066-a77b-2f71810c0e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.320788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-69038e95-0c5c-4066-a77b-2f71810c0e11 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:02.322494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-69038e95-0c5c-4066-a77b-2f71810c0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.332423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 756/1491] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:02.337467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332463d5-6ed1-433c-8d6f-1000fa66a320 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.337467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332463d5-6ed1-433c-8d6f-1000fa66a320 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.337467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332463d5-6ed1-433c-8d6f-1000fa66a320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.358098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-332463d5-6ed1-433c-8d6f-1000fa66a320 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:02.359428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-332463d5-6ed1-433c-8d6f-1000fa66a320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.369064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 757/1492] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:02.372433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.372433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.372433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.392076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:02.394434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:02.395738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.409264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 758/1493] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:02.409264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.415531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290a2f5-86a4-4ad0-9f23-d80292a27299 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.417760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c635b4a5054b959f2a2b33eb3b56ba', 'request_id': 'req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.428618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c635b4a5054b959f2a2b33eb3b56ba', 'request_id': 'req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.435833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c635b4a5054b959f2a2b33eb3b56ba', 'request_id': 'req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.491938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.493324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.493701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.494039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.494819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:02.495237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.497122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:02.516698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 759/1494] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:02.516698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.529510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29374a-f5bd-4563-8f38-f696a63b7c3a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.534743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.535593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.535962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.536325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.537119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:02.537739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.539420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0df62-f569-4747-8c2f-c26b680fc930 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:02.546170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] The token cache is being invalidate because project 40c635b4a5054b959f2a2b33eb3b56ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:02.546665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.564103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:02.564103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 736/1495] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:02 2022] DELETE /identity/v3/projects/40c635b4a5054b959f2a2b33eb3b56ba => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:02.564103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.564103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.565061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 760/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:02.568161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.568283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdff7e-be9e-4e6f-92a1-cef112ee902e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.568558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.568901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.573248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1f770-3ba3-44ba-b575-2feaced2f9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.574151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1f770-3ba3-44ba-b575-2feaced2f9eb admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.574599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1f770-3ba3-44ba-b575-2feaced2f9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.575244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1f770-3ba3-44ba-b575-2feaced2f9eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.576352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1f770-3ba3-44ba-b575-2feaced2f9eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.579048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1f770-3ba3-44ba-b575-2feaced2f9eb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:02.593619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:02.594841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.605139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 761/1497] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:02.606862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.610416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.622868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a136c-90fd-4648-8821-01841aa00c60 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.627860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.628639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.628908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.629298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.630098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.631994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:02.718319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 737/1498] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:02.723059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.723059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.723059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.747630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:02.749284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:02.781453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 762/1499] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:02.782718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.823469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 738/1500] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:02.828521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:02.857512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228caba-6e51-4b51-94c7-7293878a556b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.880696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a377c2-8e4f-4c36-bc73-30927f926cde admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:02.886003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.886634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.886902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.887184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.887810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.890351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:02.971158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:02.971673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:02.972180 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:02.972676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:02.973890 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:02.978254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.020294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:03.027525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 763/1501] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:03.027525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.039693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201d70b-67fa-4edf-89ca-d633b71a52e8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.046041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.046784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.047067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.047336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.048014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:03.048305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.050062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.067228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 764/1502] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:03.067228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.075119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859e6ab-5434-496b-a6f6-7ace87664142 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.080436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.081112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.081384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.082410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.083156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.085184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.101962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 765/1503] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:03.106842 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.121444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abcdb9-35dd-46ba-8214-209b8bb30612 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.129923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 739/1504] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:02 2022] POST /identity/v3/users => generated 287 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:03.131516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.132050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.132688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.133080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.133361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.134007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.135969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.150066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4286cca-abd8-4c75-afb4-07be609b9dc2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.158794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.159665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.159983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.160212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.161060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.162768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.178351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 740/1505] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:03.178351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.192035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fc192-0a07-494a-92e2-6b72e6245124 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.207463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.211598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.211598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.211598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.211598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.211598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.237181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 741/1506] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:03.238058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.251181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 766/1507] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:03 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:03.251181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.273111 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb7f06-d7f5-4f05-b3ff-1c9e876cb9f1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.369996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.370875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.371150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.371439 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] PATH_INFO: `/v3/projects/064e89aabf5c49c4ba93d37b18f77c6a/users/8f51f1953d92435696cc7e6100c7598e/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.389599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=064e89aabf5c49c4ba93d37b18f77c6a, user_id=8f51f1953d92435696cc7e6100c7598e, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.393816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.406906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf39cbb-356d-4bc2-8370-b4f630692b94 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.411937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.412543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.412823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.413129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.413775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.415405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.427597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 767/1508] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:03.431015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.443215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880017-3f19-4e3d-9c2e-9a7de3573a14 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.448777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b150c-e35d-471c-9205-50db8cff267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.449637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b150c-e35d-471c-9205-50db8cff267f admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.449915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b150c-e35d-471c-9205-50db8cff267f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.450185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b150c-e35d-471c-9205-50db8cff267f admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.450828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b150c-e35d-471c-9205-50db8cff267f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.453866 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b150c-e35d-471c-9205-50db8cff267f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.551645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 742/1509] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:03 2022] PUT /identity/v3/projects/064e89aabf5c49c4ba93d37b18f77c6a/users/8f51f1953d92435696cc7e6100c7598e/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:03.552719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.598559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 768/1510] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:03 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:03.599849 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffd77-c150-4cd4-b618-7d359d66e5f5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.600136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffd77-c150-4cd4-b618-7d359d66e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.600539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffd77-c150-4cd4-b618-7d359d66e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.635565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-fedffd77-c150-4cd4-b618-7d359d66e5f5 None None] MFA Rules not processed for user `8f51f1953d92435696cc7e6100c7598e`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:03.636849 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fedffd77-c150-4cd4-b618-7d359d66e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:03.649808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d22149-1908-48b4-a0bf-315de5b00b9e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.655642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff95a7c-56e0-43d9-9fdb-d5a95f72ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.656840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff95a7c-56e0-43d9-9fdb-d5a95f72ec3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.657210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff95a7c-56e0-43d9-9fdb-d5a95f72ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.657552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff95a7c-56e0-43d9-9fdb-d5a95f72ec3a admin admin] PATH_INFO: `/v3/projects/1dc1563bb2d24bdb894d96a64fbb2159/users/c5eb6def3d5d4bcfb84318382a2f2b97/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.677640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff95a7c-56e0-43d9-9fdb-d5a95f72ec3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc1563bb2d24bdb894d96a64fbb2159, user_id=c5eb6def3d5d4bcfb84318382a2f2b97, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.680662 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff95a7c-56e0-43d9-9fdb-d5a95f72ec3a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.718681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 769/1511] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:03.720183 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7197b6e-5f83-4dd8-9765-6f55a1df2890 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.720476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7197b6e-5f83-4dd8-9765-6f55a1df2890 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.720749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7197b6e-5f83-4dd8-9765-6f55a1df2890 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.722437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 770/1512] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:40:03.723166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7197b6e-5f83-4dd8-9765-6f55a1df2890 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.738186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7197b6e-5f83-4dd8-9765-6f55a1df2890 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.742947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.744021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.744309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.744580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.745516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.747158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.760250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 771/1513] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:03.761189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.771777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c416b-7f73-44bf-ba7b-4806309fde12 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:03.776472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5669e6-045b-414b-babb-101f52172a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:03.777100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5669e6-045b-414b-babb-101f52172a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.777379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5669e6-045b-414b-babb-101f52172a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.777662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5669e6-045b-414b-babb-101f52172a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.778416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5669e6-045b-414b-babb-101f52172a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:03.782041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5669e6-045b-414b-babb-101f52172a19 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:03.832133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 743/1514] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:03 2022] PUT /identity/v3/projects/1dc1563bb2d24bdb894d96a64fbb2159/users/c5eb6def3d5d4bcfb84318382a2f2b97/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:03.833920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.834195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.834466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:03.857305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:03.859328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:03.934048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 744/1515] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:03.934824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:03.996095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 772/1516] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:03 2022] POST /identity/v3/projects => generated 367 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:03.998703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:03.999220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:03.999751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.011023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01e319-519e-46a5-951f-3e9686196b7b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.058113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] MFA Rules not processed for user `c5eb6def3d5d4bcfb84318382a2f2b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:04.061906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:04.127181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.127902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.128249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.128758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.129448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.131549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.144357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 745/1517] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:04.145367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.148509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:04.179329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 773/1518] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:04.180252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.198923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d8726-659d-43af-8e62-03113ba7e414 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.205874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.206539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.206819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.207313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.208122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.210954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.239680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd22ed4-3860-466e-9a50-abe9718fa28a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.245419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.246038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.246313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.246578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.256214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.258165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.269860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 746/1519] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:04.270698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.291718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2154f-a8c6-45f7-a10e-464b685304be service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.292887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.294484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.294763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.295031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] PATH_INFO: `/v3/projects/a571bd0f24694225a2c1ae46c290e5d3/users/b2a0d36d04ed47bfaa1e849c466c72c4/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.312425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a571bd0f24694225a2c1ae46c290e5d3, user_id=b2a0d36d04ed47bfaa1e849c466c72c4, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.316121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.351547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 774/1520] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:04 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:04.351547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.352340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:04.417874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa1e83-f0f3-4644-8c36-c1ecf68d4510 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.421653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0a19a-03cb-4d70-9862-c44475916b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.422267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0a19a-03cb-4d70-9862-c44475916b85 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.422565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0a19a-03cb-4d70-9862-c44475916b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.422835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0a19a-03cb-4d70-9862-c44475916b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.432607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0a19a-03cb-4d70-9862-c44475916b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.434246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 747/1521] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:04 2022] PUT /identity/v3/projects/a571bd0f24694225a2c1ae46c290e5d3/users/b2a0d36d04ed47bfaa1e849c466c72c4/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:04.434338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0a19a-03cb-4d70-9862-c44475916b85 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.444833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.450395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 775/1522] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:04.451884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.452221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.452497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.507683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] MFA Rules not processed for user `b2a0d36d04ed47bfaa1e849c466c72c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:04.507683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:04.539174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c81f3-3650-48e2-bcae-fc2a233f63d3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.544063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.544674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.544953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.545406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.546065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.547821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.559317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 748/1523] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:04.560163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.571517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822970c-aa47-4e75-84a6-f86674d46e19 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.583535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.584270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.584452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.584726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] PATH_INFO: `/v3/users/c5eb6def3d5d4bcfb84318382a2f2b97` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.586200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5eb6def3d5d4bcfb84318382a2f2b97)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.588202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.636402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 776/1524] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:04.637444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.648655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd97bf0-4faf-42ee-a417-df920ba3af53 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.653439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.654122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.654432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.655262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] PATH_INFO: `/v3/projects/1e17dfe307ed4fcebb9e28a59817147b/users/7070876fc84a4aff9da28435a248af90/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.672774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e17dfe307ed4fcebb9e28a59817147b, user_id=7070876fc84a4aff9da28435a248af90, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.675767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.757303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:04.769180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 777/1525] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:04 2022] PUT /identity/v3/projects/1e17dfe307ed4fcebb9e28a59817147b/users/7070876fc84a4aff9da28435a248af90/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:04.770259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.811897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5eb6def3d5d4bcfb84318382a2f2b97'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:04.825691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5eb6def3d5d4bcfb84318382a2f2b97'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:04.840622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597bd84-4178-49ef-b3dd-4bd1b9372051 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:04.846043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea236f-507c-448f-b7c3-6a3bb89fed1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:04.846692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea236f-507c-448f-b7c3-6a3bb89fed1f admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:04.846961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea236f-507c-448f-b7c3-6a3bb89fed1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:04.847235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea236f-507c-448f-b7c3-6a3bb89fed1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:04.847866 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea236f-507c-448f-b7c3-6a3bb89fed1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:04.849833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea236f-507c-448f-b7c3-6a3bb89fed1f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:04.916100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5eb6def3d5d4bcfb84318382a2f2b97'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:04.916464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:04.916742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5eb6def3d5d4bcfb84318382a2f2b97'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:04.917392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5eb6def3d5d4bcfb84318382a2f2b97'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:04.930725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 749/1526] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:04 2022] DELETE /identity/v3/users/c5eb6def3d5d4bcfb84318382a2f2b97 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:04.931552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:04.932898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:04.999043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 778/1527] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:04 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:04.999834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba5ad9-21a0-44b4-a5bf-10ab03394cac admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.009324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.009324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.009324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.021448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:05.022940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.112106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.112980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.113435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.113710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.114476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.116858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.125873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 779/1528] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:05.126986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.128532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.133246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 750/1529] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:05.135877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.139843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.148628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b4c4-842e-44f5-8889-c82156a6f04c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.153582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.154513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.154513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.154817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.155394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.157589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.215069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce86109-821a-42a1-a5a9-15a1689d1426 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.220395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.221167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.221515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.221856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.232253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.234710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.245320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 751/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:05.246040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.257872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776150f-f47d-4b74-ad9e-dc6fc1788654 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.262970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6b77a-2cd4-4f67-9acd-6af586446d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.263771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6b77a-2cd4-4f67-9acd-6af586446d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.264313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6b77a-2cd4-4f67-9acd-6af586446d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.264659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6b77a-2cd4-4f67-9acd-6af586446d37 admin admin] PATH_INFO: `/v3/projects/1e17dfe307ed4fcebb9e28a59817147b/users/7070876fc84a4aff9da28435a248af90/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.268154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6b77a-2cd4-4f67-9acd-6af586446d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e17dfe307ed4fcebb9e28a59817147b, user_id=7070876fc84a4aff9da28435a248af90, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.271500 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6b77a-2cd4-4f67-9acd-6af586446d37 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.287052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 752/1531] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:05 2022] PUT /identity/v3/projects/1e17dfe307ed4fcebb9e28a59817147b/users/7070876fc84a4aff9da28435a248af90/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:05.298983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.299310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.299737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.319683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 780/1532] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:05 2022] POST /identity/v3/users => generated 299 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:05.324359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.337696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] MFA Rules not processed for user `7070876fc84a4aff9da28435a248af90`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:05.339156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.346304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51558f-3dc3-4863-acae-a226ec387451 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.472797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 753/1533] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:05.490035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.491709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.505267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.506041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.506391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.507078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.507484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.509504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.526146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 781/1534] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:05.527492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.561860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7226ca-b4fa-4ef3-93ac-58a93d5d86f1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.575272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff87c74-38df-4c82-a550-07635c475f4e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.576074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.580913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.581328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.581585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.582373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] PATH_INFO: `/v3/projects/f43250d2bab44e188218b4ccc49fb8ab/users/dd3c9d45da464a818317fc8bd5928b59/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.583277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.583920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.584352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.602267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43250d2bab44e188218b4ccc49fb8ab, user_id=dd3c9d45da464a818317fc8bd5928b59, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.606302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.617352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.619591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.632576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 754/1535] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:05.747572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 782/1536] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:05 2022] PUT /identity/v3/projects/f43250d2bab44e188218b4ccc49fb8ab/users/dd3c9d45da464a818317fc8bd5928b59/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:05.751464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.762416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13cca-86cb-4d78-9d8c-736b2ce1e528 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.838458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.839505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.839874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.840248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.841247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.849021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.861430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 755/1537] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:05.862890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.876445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351ec93-f680-4213-bdbe-aa1af2845add admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:05.884213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed083ba-69b1-4a87-8a5a-1490817ddf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:05.884213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed083ba-69b1-4a87-8a5a-1490817ddf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.884213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed083ba-69b1-4a87-8a5a-1490817ddf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.884213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed083ba-69b1-4a87-8a5a-1490817ddf3d admin admin] PATH_INFO: `/v3/projects/f43250d2bab44e188218b4ccc49fb8ab/users/dd3c9d45da464a818317fc8bd5928b59/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.885130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed083ba-69b1-4a87-8a5a-1490817ddf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43250d2bab44e188218b4ccc49fb8ab, user_id=dd3c9d45da464a818317fc8bd5928b59, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:05.887892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed083ba-69b1-4a87-8a5a-1490817ddf3d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:05.900049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:05.902111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 783/1538] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:05 2022] PUT /identity/v3/projects/f43250d2bab44e188218b4ccc49fb8ab/users/dd3c9d45da464a818317fc8bd5928b59/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:05.903136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.916664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:05.916664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:05.916939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:05.956403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] MFA Rules not processed for user `dd3c9d45da464a818317fc8bd5928b59`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:05.958541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:05.990830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24df8-13cf-4322-af86-cc637b675a5a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.068706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.069801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.072560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.072560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.073504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:06.080354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 784/1539] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:06.085439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:06.085439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:06.152833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.154420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.165589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 756/1540] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:06.178013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d32d9a-3bdd-479a-88e6-e4de245405ec None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.181942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.182582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.182856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.183132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] PATH_INFO: `/v3/projects/1dc1563bb2d24bdb894d96a64fbb2159` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.184257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc1563bb2d24bdb894d96a64fbb2159)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.186391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.231213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:06.240647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9fc2-7ac7-4fe3-8d95-13b5cd63bc36 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.244456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c3ca-01f6-46e3-a6cb-6c967db8b576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.246135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c3ca-01f6-46e3-a6cb-6c967db8b576 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.246135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c3ca-01f6-46e3-a6cb-6c967db8b576 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.246135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c3ca-01f6-46e3-a6cb-6c967db8b576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.258640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5c3ca-01f6-46e3-a6cb-6c967db8b576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.260136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5c3ca-01f6-46e3-a6cb-6c967db8b576 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.276227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 757/1541] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:06.357213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc1563bb2d24bdb894d96a64fbb2159', 'request_id': 'req-f844f6e3-f424-4de9-8ff0-4c148e5874a8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:06.364984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc1563bb2d24bdb894d96a64fbb2159', 'request_id': 'req-f844f6e3-f424-4de9-8ff0-4c148e5874a8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:06.425895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc1563bb2d24bdb894d96a64fbb2159', 'request_id': 'req-f844f6e3-f424-4de9-8ff0-4c148e5874a8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:06.448451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] The token cache is being invalidate because project 1dc1563bb2d24bdb894d96a64fbb2159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:06.448451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:06.448451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:06.448451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 785/1542] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:06 2022] DELETE /identity/v3/projects/1dc1563bb2d24bdb894d96a64fbb2159 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:06.453881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e15827-ea86-41ba-8437-1a29daed5068 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.454226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e15827-ea86-41ba-8437-1a29daed5068 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.455337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e15827-ea86-41ba-8437-1a29daed5068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.480306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d3e15827-ea86-41ba-8437-1a29daed5068 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:06.480306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d3e15827-ea86-41ba-8437-1a29daed5068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:06.512386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:06.513905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:06.559053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 758/1543] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:06.567570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.567854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.568122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.598320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:06.600786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:06.602097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f6e3-f424-4de9-8ff0-4c148e5874a8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.607453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.608710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.609196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.609614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] PATH_INFO: `/v3/users/b2a0d36d04ed47bfaa1e849c466c72c4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.610896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a0d36d04ed47bfaa1e849c466c72c4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.613089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.620063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 759/1544] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:06.622371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:06.647955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53e0de-5774-494a-83fb-5a512ebd7e43 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.651967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.652577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.652842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.653175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.655287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:06.655774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.658085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.672844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 760/1545] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:06.676247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:06.687423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d875ef-4db0-4b68-897e-eee1da5afb62 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.692361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.692979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.693296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.693573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.694214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.696170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.853648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 761/1546] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:06 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:06.860412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:06.916938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a0d36d04ed47bfaa1e849c466c72c4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:06.961405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43f0d-6198-4807-9fb5-674945bc065b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:06.974378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:06.975335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:06.975826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:06.976473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:06.977998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:06.981384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:06.997254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a0d36d04ed47bfaa1e849c466c72c4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:07.013049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a0d36d04ed47bfaa1e849c466c72c4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:07.014426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a0d36d04ed47bfaa1e849c466c72c4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:07.015168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d9f0f714-ec97-4f1f-977e-739e471ebd00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a0d36d04ed47bfaa1e849c466c72c4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:07.031447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 786/1547] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:06 2022] DELETE /identity/v3/users/b2a0d36d04ed47bfaa1e849c466c72c4 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:07.035898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.036397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.036668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.056817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:07.058099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:07.136813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 787/1548] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:07.147102 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 762/1549] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:06 2022] POST /identity/v3/users => generated 286 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:07.147931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:07.149593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:07.149710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:07.152423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:07.167410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ec59b-5d20-4886-ad8f-4a3d6fa374c6 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:07.173635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:07.174266 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.174551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.174825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.175479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:07.177307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:07.188596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 788/1550] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:07.192589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:07.203311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddbb7-ad1d-41ac-bf12-06dab68f6fe1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:07.208305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:07.208918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.209238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.209510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.210203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:07.211852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:07.225024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 789/1551] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:07.230149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:07.236544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbec9e8-369c-4959-8452-b86bbb1811ca admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:07.242252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242e9f-7867-4d66-95a8-b86ebf1cefab admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:07.249587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:07.250843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.251376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.251961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] PATH_INFO: `/v3/projects/0c51788c68ee45b99646d4699d17a28c/users/8fd9ccfc40744fa486d7a34d3b7dabbf/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.271129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c51788c68ee45b99646d4699d17a28c, user_id=8fd9ccfc40744fa486d7a34d3b7dabbf, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:07.274635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:07.303037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931aff5-63e5-4d50-89f8-1012c4dcbb94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:07.303644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931aff5-63e5-4d50-89f8-1012c4dcbb94 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.303989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931aff5-63e5-4d50-89f8-1012c4dcbb94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.304209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931aff5-63e5-4d50-89f8-1012c4dcbb94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.314446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931aff5-63e5-4d50-89f8-1012c4dcbb94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:07.315927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931aff5-63e5-4d50-89f8-1012c4dcbb94 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:07.327648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 763/1552] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:07.373934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 790/1553] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:07 2022] PUT /identity/v3/projects/0c51788c68ee45b99646d4699d17a28c/users/8fd9ccfc40744fa486d7a34d3b7dabbf/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:07.375614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58212635-e8be-426e-bae7-7f5552200026 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.375892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58212635-e8be-426e-bae7-7f5552200026 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.376164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58212635-e8be-426e-bae7-7f5552200026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.410560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-58212635-e8be-426e-bae7-7f5552200026 None None] MFA Rules not processed for user `8fd9ccfc40744fa486d7a34d3b7dabbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:07.419482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-58212635-e8be-426e-bae7-7f5552200026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:07.510486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 764/1554] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:07.524565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:07.526406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:07.635703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7aef4-4016-43c1-b4f0-9c3a80b7bb3a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:07.639972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:07.646897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.646897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.646897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.656099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:07.663271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:07.681589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 791/1555] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:07.864526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58212635-e8be-426e-bae7-7f5552200026 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:07.864526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-58212635-e8be-426e-bae7-7f5552200026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:07.971732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58212635-e8be-426e-bae7-7f5552200026 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:07.977062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:07.978346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:07.978825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:07.979245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] PATH_INFO: `/v3/projects/a571bd0f24694225a2c1ae46c290e5d3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:07.981373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a571bd0f24694225a2c1ae46c290e5d3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:07.983880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:08.176589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a571bd0f24694225a2c1ae46c290e5d3', 'request_id': 'req-0bb3f501-2fc3-4506-a6df-a94934e1fa62'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.193660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a571bd0f24694225a2c1ae46c290e5d3', 'request_id': 'req-0bb3f501-2fc3-4506-a6df-a94934e1fa62'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.206737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a571bd0f24694225a2c1ae46c290e5d3', 'request_id': 'req-0bb3f501-2fc3-4506-a6df-a94934e1fa62'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.326839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] The token cache is being invalidate because project a571bd0f24694225a2c1ae46c290e5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:08.327186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.327888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.329883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 765/1556] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:07 2022] DELETE /identity/v3/projects/a571bd0f24694225a2c1ae46c290e5d3 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:08.340476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:08.354302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2cfde-4d2d-4e3d-be0a-9665dccfffa9 service glance] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:08.445050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:08.445703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:08.446008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:08.446326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] PATH_INFO: `/v3/users/dd3c9d45da464a818317fc8bd5928b59` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:08.454082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3c9d45da464a818317fc8bd5928b59)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:08.454082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:08.476839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:08.712064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:08.815949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3c9d45da464a818317fc8bd5928b59'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.927736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3c9d45da464a818317fc8bd5928b59'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.941709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3c9d45da464a818317fc8bd5928b59'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.942476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3c9d45da464a818317fc8bd5928b59'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.946720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a6f3be4c-6141-4a75-9ec7-f8902e54da52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3c9d45da464a818317fc8bd5928b59'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:08.956888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 792/1557] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:08 2022] DELETE /identity/v3/users/dd3c9d45da464a818317fc8bd5928b59 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:09.560367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:09.563268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:09.582560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f501-2fc3-4506-a6df-a94934e1fa62 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:09.681839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:09.682545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:09.682824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:09.683157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] PATH_INFO: `/v3/projects/f43250d2bab44e188218b4ccc49fb8ab` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:09.684374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43250d2bab44e188218b4ccc49fb8ab)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:09.687067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:09.809465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43250d2bab44e188218b4ccc49fb8ab', 'request_id': 'req-fa66b6f4-27ca-4a6d-a79b-c336d198731a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:09.815841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43250d2bab44e188218b4ccc49fb8ab', 'request_id': 'req-fa66b6f4-27ca-4a6d-a79b-c336d198731a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:09.821299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43250d2bab44e188218b4ccc49fb8ab', 'request_id': 'req-fa66b6f4-27ca-4a6d-a79b-c336d198731a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:09.939980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] The token cache is being invalidate because project f43250d2bab44e188218b4ccc49fb8ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:09.940496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:09.941123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fa66b6f4-27ca-4a6d-a79b-c336d198731a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:09.942848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 766/1558] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:09 2022] DELETE /identity/v3/projects/f43250d2bab44e188218b4ccc49fb8ab => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:09.961847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:09.962683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:09.962955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:09.991756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:09.992990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:10.055504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 793/1559] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:10.066017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.066305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.066610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.087824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:10.089174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:10.099114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 767/1560] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:10.105758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:10.122493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be8b94-7f5b-4bd7-a0dd-80c29d16a275 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:10.130515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:10.131498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.131981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.132782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.133610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:10.134084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:10.136801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:10.159143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 794/1561] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:10.160126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:10.169590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dbcef-0a4b-4183-bcc2-0c0b718a7c24 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:10.175662 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:10.176299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.176603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.176845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.177550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:10.180731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:10.267587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:10.275168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 768/1562] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:10 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:10.279780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:10.296200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305bcc-c978-4db7-b40d-b40d545a7b34 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:10.357809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:10.358415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.358689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.358956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.359591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:10.361585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:10.505903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 795/1563] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:10 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:10.510040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:10.520926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f03a7-3099-4467-b84c-8d316ade8c6a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:10.525650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:10.526261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.526531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.526802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.527449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:10.529164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:10.543532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 769/1564] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:10.546433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:10.556760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9c394-5a01-4f4c-b8dc-80c7c9cf6300 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:10.561969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e9f8d-7068-4bbf-af5b-966eb605754b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:10.562581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9f8d-7068-4bbf-af5b-966eb605754b admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.562857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9f8d-7068-4bbf-af5b-966eb605754b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.563128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9f8d-7068-4bbf-af5b-966eb605754b admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.563771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e9f8d-7068-4bbf-af5b-966eb605754b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:10.565486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e9f8d-7068-4bbf-af5b-966eb605754b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:10.595170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 796/1565] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:10.598329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:10.609372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab9b9-d609-4710-b747-86560416805f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:10.620050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:10.621398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.621988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.622573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] PATH_INFO: `/v3/projects/ceb6d5903f944f3abc8b390e37bb74ba/users/12ddea07b96d4a3e8070f1fed680e1cc/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.659692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb6d5903f944f3abc8b390e37bb74ba, user_id=12ddea07b96d4a3e8070f1fed680e1cc, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:10.662517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:10.809211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 770/1566] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:10 2022] PUT /identity/v3/projects/ceb6d5903f944f3abc8b390e37bb74ba/users/12ddea07b96d4a3e8070f1fed680e1cc/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:10.820527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:10.820817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:10.821125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:10.851354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] MFA Rules not processed for user `12ddea07b96d4a3e8070f1fed680e1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:10.852479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:10.928530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 797/1567] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:10.933259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:11.003576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63515229-46a7-4d2a-99d8-10c42b6ff7be admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:11.008857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:11.009675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.010293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.010582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.011304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:11.018140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:11.164425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 771/1568] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:10 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:11.168908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:11.182497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab2d02-d73c-4c7c-a387-17b7d7755b6f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:11.249856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:11.261206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:11.261206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.261206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.261206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.261627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:11.261627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:11.377825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 798/1569] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:11 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:11.380113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:11.393736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce077c9-eb5c-454d-9bde-5787a350b732 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:11.416792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:11.417544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.417966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.418238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.423367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:11.424456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:11.451125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 772/1570] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:11.458540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:11.476083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837734b2-5028-479d-b95b-164d2ec7b1ea admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:11.482033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:11.485450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.486673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.487092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] PATH_INFO: `/v3/projects/3acaa75603064e94a46345fa8a48a36d/users/4232af5c10cd4ab694ceb019d2dfae49/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.507515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acaa75603064e94a46345fa8a48a36d, user_id=4232af5c10cd4ab694ceb019d2dfae49, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:11.510397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:11.617640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 799/1571] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:11 2022] PUT /identity/v3/projects/3acaa75603064e94a46345fa8a48a36d/users/4232af5c10cd4ab694ceb019d2dfae49/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:11.621344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:11.634292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9f66-d0c3-46c6-9b2e-4e2a452a2d56 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:11.725746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166c71a-5fe3-4559-b3aa-7edbe812ec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:11.726646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166c71a-5fe3-4559-b3aa-7edbe812ec17 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.727063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166c71a-5fe3-4559-b3aa-7edbe812ec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.727335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166c71a-5fe3-4559-b3aa-7edbe812ec17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.728232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166c71a-5fe3-4559-b3aa-7edbe812ec17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:11.729995 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166c71a-5fe3-4559-b3aa-7edbe812ec17 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:11.748953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 773/1572] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:11.750144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:11.767997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e914b-23e7-40d7-aed9-f199af7c0a7a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:11.773969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:11.773969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.774215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.774523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] PATH_INFO: `/v3/projects/3acaa75603064e94a46345fa8a48a36d/users/4232af5c10cd4ab694ceb019d2dfae49/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.776706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acaa75603064e94a46345fa8a48a36d, user_id=4232af5c10cd4ab694ceb019d2dfae49, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:11.779875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:11.793101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 800/1573] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:11 2022] PUT /identity/v3/projects/3acaa75603064e94a46345fa8a48a36d/users/4232af5c10cd4ab694ceb019d2dfae49/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:11.805075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:11.805335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:11.805583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:11.853836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] MFA Rules not processed for user `4232af5c10cd4ab694ceb019d2dfae49`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:11.855346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:11.983279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 774/1574] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:12.004009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:12.005968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:12.097905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee9900-07bb-40be-b056-8a00d0bace3f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:12.118579 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:12.120552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:12.122727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:12.124107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:12.157200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:12.159440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:12.189818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 801/1575] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:12.777199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:12.838425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:14.932656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:14.942678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:15.029170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74e35e-bbe7-47ca-abbd-7864e01030c4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:15.034263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:15.034946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:15.035224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:15.035499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] PATH_INFO: `/v3/users/12ddea07b96d4a3e8070f1fed680e1cc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:15.036873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ddea07b96d4a3e8070f1fed680e1cc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:15.038955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:15.478616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ddea07b96d4a3e8070f1fed680e1cc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:15.502281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:15.502281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:15.532413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ddea07b96d4a3e8070f1fed680e1cc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:15.537385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7631215-1aff-4019-a8f7-0f15b1b6a5ac service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:15.550558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:15.551562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:15.552204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:15.553085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] PATH_INFO: `/v3/users/7fd21cedfb844c3893b936ab6cf5b047` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:15.555512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd21cedfb844c3893b936ab6cf5b047)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:15.562142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:15.704609 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ddea07b96d4a3e8070f1fed680e1cc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:15.705479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ddea07b96d4a3e8070f1fed680e1cc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:15.706138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-333464d5-1d28-4020-b8f0-ef37e1c05a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ddea07b96d4a3e8070f1fed680e1cc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:15.724065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 775/1576] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:14 2022] DELETE /identity/v3/users/12ddea07b96d4a3e8070f1fed680e1cc => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:15.728848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:15.729165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:15.729443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:15.750225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:15.751469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:15.833240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd21cedfb844c3893b936ab6cf5b047'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:15.848643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 776/1577] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:15.866146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:15.867769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:15.979025 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5d06-a109-43f9-bfba-cc6f3a6b882f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:15.988763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af7bad-7291-4d4e-92bd-ad1578b5602e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:15.990043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af7bad-7291-4d4e-92bd-ad1578b5602e service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:15.990678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af7bad-7291-4d4e-92bd-ad1578b5602e service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:15.991291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af7bad-7291-4d4e-92bd-ad1578b5602e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:16.009321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd21cedfb844c3893b936ab6cf5b047'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.025923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af7bad-7291-4d4e-92bd-ad1578b5602e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:16.027679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af7bad-7291-4d4e-92bd-ad1578b5602e service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:16.048667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd21cedfb844c3893b936ab6cf5b047'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.049901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd21cedfb844c3893b936ab6cf5b047'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.051203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd21cedfb844c3893b936ab6cf5b047'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.056196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 777/1578] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:16.071903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 802/1579] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:15 2022] DELETE /identity/v3/users/7fd21cedfb844c3893b936ab6cf5b047 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:16.079291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:16.079752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:16.080139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:16.118685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:16.120446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:16.208400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 778/1580] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:16.218958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:16.221212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:16.281916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:16.316326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fc7ee-ad9d-4a2a-84cf-d47c0ddadb68 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:16.320215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:16.320843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:16.321294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:16.321568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:16.337822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:16.339284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:16.350207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 803/1581] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:16.459744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:16.461186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:16.473673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acda38-1e3e-4fdc-b85a-c780498e9c72 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:16.480408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:16.481608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:16.482155 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:16.482644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] PATH_INFO: `/v3/users/8f51f1953d92435696cc7e6100c7598e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:16.485690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f51f1953d92435696cc7e6100c7598e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:16.488428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:16.609138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:16.612110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:16.652806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1566e1-970b-4d01-a304-3a6302824f8e service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:16.660398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:16.661306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:16.661648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:16.662848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] PATH_INFO: `/v3/projects/ceb6d5903f944f3abc8b390e37bb74ba` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:16.665846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb6d5903f944f3abc8b390e37bb74ba)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:16.669313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:16.728524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f51f1953d92435696cc7e6100c7598e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.756324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f51f1953d92435696cc7e6100c7598e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.788531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb6d5903f944f3abc8b390e37bb74ba', 'request_id': 'req-ac1137ac-a6e2-43fe-b10f-2033819f73f4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.795576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb6d5903f944f3abc8b390e37bb74ba', 'request_id': 'req-ac1137ac-a6e2-43fe-b10f-2033819f73f4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.838348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f51f1953d92435696cc7e6100c7598e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.839464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f51f1953d92435696cc7e6100c7598e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.840449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f51f1953d92435696cc7e6100c7598e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.862358 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 779/1582] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:16 2022] DELETE /identity/v3/users/8f51f1953d92435696cc7e6100c7598e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:16.867415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:16.867415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:16.922302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb6d5903f944f3abc8b390e37bb74ba', 'request_id': 'req-ac1137ac-a6e2-43fe-b10f-2033819f73f4'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.937511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] The token cache is being invalidate because project ceb6d5903f944f3abc8b390e37bb74ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:16.938035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.938879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ac1137ac-a6e2-43fe-b10f-2033819f73f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:16.940883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 804/1583] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:16 2022] DELETE /identity/v3/projects/ceb6d5903f944f3abc8b390e37bb74ba => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:16.942480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:16.942935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:16.943518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:16.973119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:16.974694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:17.002150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fb2aa-9ba9-44b4-9365-c4c9a1081cd3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:17.008181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:17.012698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:17.012698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:17.012698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] PATH_INFO: `/v3/projects/ad47caf1520a49dc83d4bc12455a854e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:17.012698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad47caf1520a49dc83d4bc12455a854e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:17.015424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:17.086499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 805/1584] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:17.089493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:17.090820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:17.106222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f58e2-09a3-4656-99ae-d50c2910e10f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:17.120803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:17.121591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:17.121963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:17.122299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] PATH_INFO: `/v3/users/4232af5c10cd4ab694ceb019d2dfae49` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:17.126305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4232af5c10cd4ab694ceb019d2dfae49)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:17.129965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:17.143887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad47caf1520a49dc83d4bc12455a854e', 'request_id': 'req-2eaf8120-366f-4405-98d0-30a6f4933215'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.151561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad47caf1520a49dc83d4bc12455a854e', 'request_id': 'req-2eaf8120-366f-4405-98d0-30a6f4933215'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.164900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad47caf1520a49dc83d4bc12455a854e', 'request_id': 'req-2eaf8120-366f-4405-98d0-30a6f4933215'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.272227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] The token cache is being invalidate because project ad47caf1520a49dc83d4bc12455a854e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:17.272517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.273252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.274844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 780/1585] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:16 2022] DELETE /identity/v3/projects/ad47caf1520a49dc83d4bc12455a854e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:17.275707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:17.277713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:17.343859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf8120-366f-4405-98d0-30a6f4933215 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:17.347758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:17.348393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:17.348892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:17.349260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:17.351048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:17.382038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4232af5c10cd4ab694ceb019d2dfae49'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.431041 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:17.432578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:17.450619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 781/1586] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:17.451567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:17.453094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:17.469079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de312af7-506a-40d1-9fee-0c1b6bb66ba6 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:17.474448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:17.475955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:17.475955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:17.475955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] PATH_INFO: `/v3/users/d4051ad4e8bf44dd9ea868a8bfdc9267` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:17.477135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4051ad4e8bf44dd9ea868a8bfdc9267)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:17.479644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:17.505206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4232af5c10cd4ab694ceb019d2dfae49'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.527456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4232af5c10cd4ab694ceb019d2dfae49'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.528557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4232af5c10cd4ab694ceb019d2dfae49'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.529893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4232af5c10cd4ab694ceb019d2dfae49'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.552338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 806/1587] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:17 2022] DELETE /identity/v3/users/4232af5c10cd4ab694ceb019d2dfae49 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:17.729533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4051ad4e8bf44dd9ea868a8bfdc9267'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.769583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4051ad4e8bf44dd9ea868a8bfdc9267'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.888075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4051ad4e8bf44dd9ea868a8bfdc9267'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.889234 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4051ad4e8bf44dd9ea868a8bfdc9267'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.891394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4051ad4e8bf44dd9ea868a8bfdc9267'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:17.909371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 782/1588] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:17 2022] DELETE /identity/v3/users/d4051ad4e8bf44dd9ea868a8bfdc9267 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:18.032746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:18.035128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:18.128075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:18.194002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c796210-675e-44b3-b8fb-4f90c154652f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:18.306729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:18.307370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:18.307647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:18.307917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] PATH_INFO: `/v3/projects/064e89aabf5c49c4ba93d37b18f77c6a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:18.310910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=064e89aabf5c49c4ba93d37b18f77c6a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:18.312864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:18.479565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064e89aabf5c49c4ba93d37b18f77c6a', 'request_id': 'req-53abee03-1abf-4d84-baf8-7dc7624dbd8d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.487109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064e89aabf5c49c4ba93d37b18f77c6a', 'request_id': 'req-53abee03-1abf-4d84-baf8-7dc7624dbd8d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.562966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064e89aabf5c49c4ba93d37b18f77c6a', 'request_id': 'req-53abee03-1abf-4d84-baf8-7dc7624dbd8d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.585819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:18.587951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:18.588956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] The token cache is being invalidate because project 064e89aabf5c49c4ba93d37b18f77c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:18.589653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.599698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.599698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 807/1589] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:18 2022] DELETE /identity/v3/projects/064e89aabf5c49c4ba93d37b18f77c6a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:18.603168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:18.604740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c169e0e-cfe6-4488-9d54-ead429b936c4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:18.605302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:18.692498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:18.693190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:18.693477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:18.693751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] PATH_INFO: `/v3/projects/3acaa75603064e94a46345fa8a48a36d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:18.695909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3acaa75603064e94a46345fa8a48a36d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:18.697270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:18.705604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abee03-1abf-4d84-baf8-7dc7624dbd8d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:18.712429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:18.713054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:18.714031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:18.714475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] PATH_INFO: `/v3/users/7070876fc84a4aff9da28435a248af90` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:18.716440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7070876fc84a4aff9da28435a248af90)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:18.718774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:18.944253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7070876fc84a4aff9da28435a248af90'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.948146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3acaa75603064e94a46345fa8a48a36d', 'request_id': 'req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.955358 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3acaa75603064e94a46345fa8a48a36d', 'request_id': 'req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.961823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3acaa75603064e94a46345fa8a48a36d', 'request_id': 'req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:18.985209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:19.035551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7070876fc84a4aff9da28435a248af90'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.055438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7070876fc84a4aff9da28435a248af90'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.057197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7070876fc84a4aff9da28435a248af90'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.058039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7070876fc84a4aff9da28435a248af90'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.071874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:19.097279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 808/1590] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:18 2022] DELETE /identity/v3/users/7070876fc84a4aff9da28435a248af90 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:19.098332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.100471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.153452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] The token cache is being invalidate because project 3acaa75603064e94a46345fa8a48a36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:19.153452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.153452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f0102a3c-4fb2-49ee-a3b9-db0b76df290c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.153452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 783/1591] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:18 2022] DELETE /identity/v3/projects/3acaa75603064e94a46345fa8a48a36d => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:19.153452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c527cd-0339-4dc3-95b7-708d3806712e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.155001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c527cd-0339-4dc3-95b7-708d3806712e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.155001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c527cd-0339-4dc3-95b7-708d3806712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.186331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-61c527cd-0339-4dc3-95b7-708d3806712e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:19.187614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-61c527cd-0339-4dc3-95b7-708d3806712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.240595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fdbf1-fc09-4b85-bb2a-a4fab7746ee1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.252632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.253319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.253599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.254067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] PATH_INFO: `/v3/projects/8e051c99570e4bb1bb57e186ee5f566c` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.255363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e051c99570e4bb1bb57e186ee5f566c)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.257439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.286308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 784/1592] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:19.293813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.294100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.294376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.315079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:19.316295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.324371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 785/1593] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:19.329483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.338618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcac3b-b595-4ac9-8a82-e0a3a44b1dc4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.343622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.344227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.344503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.344784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.345642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:19.345954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.347567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.360176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 786/1594] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:40:19.364763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.374998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724367f-760f-4a12-985c-b1608ac1bd5d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.379991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.380550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.380801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.381127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.381756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.383687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.407028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e051c99570e4bb1bb57e186ee5f566c', 'request_id': 'req-e8b08643-2d6b-40ee-9983-d49454ded25d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.413678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e051c99570e4bb1bb57e186ee5f566c', 'request_id': 'req-e8b08643-2d6b-40ee-9983-d49454ded25d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.500245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e051c99570e4bb1bb57e186ee5f566c', 'request_id': 'req-e8b08643-2d6b-40ee-9983-d49454ded25d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.509879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 787/1595] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:19.514762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.524542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] The token cache is being invalidate because project 8e051c99570e4bb1bb57e186ee5f566c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:19.525727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.525998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e8b08643-2d6b-40ee-9983-d49454ded25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.532001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 809/1596] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:19 2022] DELETE /identity/v3/projects/8e051c99570e4bb1bb57e186ee5f566c => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:19.533863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c3f1b-e814-4975-a3cb-f7e2a284d87b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.534192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c3f1b-e814-4975-a3cb-f7e2a284d87b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.534440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c3f1b-e814-4975-a3cb-f7e2a284d87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.559877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e22c3f1b-e814-4975-a3cb-f7e2a284d87b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:19.562001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e22c3f1b-e814-4975-a3cb-f7e2a284d87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.598593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d62da-1b0b-46e4-bbe0-ccbc9bfceb74 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.605249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.605998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.606274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.606544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.607213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.609517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.661410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 810/1597] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:19.671383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.671674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.671944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.682454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 788/1598] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:19.683451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.686246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.698469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:19.699689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.702202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb5840-ae71-47b1-8f14-f550e72a4d9d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.711559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 811/1599] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:19.711831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.711831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.711831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.711831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.712287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.712721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.713416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.724146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4de9d7-9c10-4451-a350-5ebb351f782f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.729410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 789/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:19.736443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.741651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.742283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.742746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.742850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.743520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:19.743838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.746146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.751164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1993a8a-0e59-4122-995d-17673f61cd8c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.756675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.757345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.757619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.760003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.760807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.762708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.766606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 812/1601] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:40:19.768019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.777127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 790/1602] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:19.777801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.782972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:19.792100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3ef34-c884-4e0d-8da5-b56ea6115a27 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.799363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.799586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d58f7-62a4-4f64-977b-9670b4a69774 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:19.801059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.801591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.801985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.802879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.804987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.806300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:19.806909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:19.811074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:19.811074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] PATH_INFO: `/v3/projects/1e17dfe307ed4fcebb9e28a59817147b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:19.812916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e17dfe307ed4fcebb9e28a59817147b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:19.814915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:19.925822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 791/1603] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:19 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:19.931550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:19.953830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e17dfe307ed4fcebb9e28a59817147b', 'request_id': 'req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:19.963437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e17dfe307ed4fcebb9e28a59817147b', 'request_id': 'req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:20.010484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d09b3-0273-4e6f-a1f9-bdb87ae55e46 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.058228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e17dfe307ed4fcebb9e28a59817147b', 'request_id': 'req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:20.074291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] The token cache is being invalidate because project 1e17dfe307ed4fcebb9e28a59817147b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:20.074600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:20.075237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:20.077380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 813/1604] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:19 2022] DELETE /identity/v3/projects/1e17dfe307ed4fcebb9e28a59817147b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:20.078307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.079591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:20.082908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0eeb0-a185-4769-99d1-acdec4f93e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.083831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0eeb0-a185-4769-99d1-acdec4f93e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.084304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0eeb0-a185-4769-99d1-acdec4f93e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.084751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0eeb0-a185-4769-99d1-acdec4f93e1e admin admin] PATH_INFO: `/v3/projects/7d90eea69e914f039443078ac05d0518/users/e509668e9328439296f7dd64c563a81f/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.093398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cda6d1-f3bc-4270-afa3-b07e7d6bae1c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.102186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1088a-3815-4c34-8131-8c82da6034bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.103482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1088a-3815-4c34-8131-8c82da6034bb admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.103943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1088a-3815-4c34-8131-8c82da6034bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.104431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1088a-3815-4c34-8131-8c82da6034bb admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.105187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1088a-3815-4c34-8131-8c82da6034bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.107507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0eeb0-a185-4769-99d1-acdec4f93e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d90eea69e914f039443078ac05d0518, user_id=e509668e9328439296f7dd64c563a81f, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.107643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1088a-3815-4c34-8131-8c82da6034bb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.110543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0eeb0-a185-4769-99d1-acdec4f93e1e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.183501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 792/1605] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:19 2022] PUT /identity/v3/projects/7d90eea69e914f039443078ac05d0518/users/e509668e9328439296f7dd64c563a81f/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:20.185738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.186329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.186780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.209040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 814/1606] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:20.210593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53099865-e9ef-4755-8b64-ee431361e149 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.211576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53099865-e9ef-4755-8b64-ee431361e149 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.212106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53099865-e9ef-4755-8b64-ee431361e149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.230098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:20.230098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:20.270601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-53099865-e9ef-4755-8b64-ee431361e149 None None] MFA Rules not processed for user `e509668e9328439296f7dd64c563a81f`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:20.272010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-53099865-e9ef-4755-8b64-ee431361e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:20.355975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 793/1607] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:20.357846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.381522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d88c7-bc2f-4e79-9e32-03c647a63d23 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.402539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 815/1608] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:20.402817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.403399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.403671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.404465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.404650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.404743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.405034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.405700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.407329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.420379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 794/1609] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:20.421994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.424545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:20.443940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ce13b-34e0-4b89-8386-15c5287be936 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.446601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:20.448366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:20.455528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.456185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.456469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.456746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.457753 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.459772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.481392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 816/1610] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:20.484322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.495318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbfcb2-d843-4e69-a9db-96beb448d7ed None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.504835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.504835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.504835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.504835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.505284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.506771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.521077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 817/1611] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:20.521077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.531301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac59b0e-18c7-459a-bcc0-a688964242cf admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.536079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.536710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.536989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.537452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.538106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:20.538409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.539991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.556756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 818/1612] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:20.556756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.564751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80406-0ecf-485d-b8f2-8c7e773e5ae0 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.569846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.570523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.570787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.571051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] PATH_INFO: `/v3/projects/5d7cbc53a96c40a99d32ab6b0c6d2f21/users/20d81ee13cb9482eaaa19ab797f1ad98/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.588449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7cbc53a96c40a99d32ab6b0c6d2f21, user_id=20d81ee13cb9482eaaa19ab797f1ad98, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.591083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.644842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 795/1613] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:20.645798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.664684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 819/1614] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:20 2022] PUT /identity/v3/projects/5d7cbc53a96c40a99d32ab6b0c6d2f21/users/20d81ee13cb9482eaaa19ab797f1ad98/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:20.665550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.724717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5f5c-81d2-475a-8805-e9e901b7d0fd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.749567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:20.768790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50204a32-0e1f-4291-a8ec-0bd29e9f7a3e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:20.776358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959f4b-ef31-464f-8989-6589f89cf0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.776358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959f4b-ef31-464f-8989-6589f89cf0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.776358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959f4b-ef31-464f-8989-6589f89cf0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.776358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959f4b-ef31-464f-8989-6589f89cf0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.776698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959f4b-ef31-464f-8989-6589f89cf0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.777884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959f4b-ef31-464f-8989-6589f89cf0c6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.796274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:20.796591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:20.796893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.797203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.797477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.798143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:20.800151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:20.860207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 820/1615] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:20.861525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:20.861816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:20.862092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:20.902580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] MFA Rules not processed for user `20d81ee13cb9482eaaa19ab797f1ad98`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:20.902580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:20.936446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 796/1616] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:20.940419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.982139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 821/1617] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:20.983167 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:20.995031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb895c-44b6-4660-b519-f18be454da0d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.023941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961d0b9-ed4b-43e4-b1fe-9d62a2f056d2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.028922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.029622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.029884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.030153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.030805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.032457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.048873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 797/1618] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:21.050047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.061150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa56e-ac3d-41ed-b80a-ef00296e8d4a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.066959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.067588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.067918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.068119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.069051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.070762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.079069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.079878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.079976 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.080212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.080852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.083237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.168463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 822/1619] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:20 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:21.170116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.184858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 798/1620] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:21 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:21.186055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966d955-ef8b-4ff5-b59a-f86f96a79d2b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.186202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.273810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.273810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.273810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.273810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] PATH_INFO: `/v3/projects/9b95ed1c6d8c42778bb91e846e63d9d9/users/1a80343ac8154e70b013c93eaf7c380a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.291067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b95ed1c6d8c42778bb91e846e63d9d9, user_id=1a80343ac8154e70b013c93eaf7c380a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.293986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.345447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaefa0-95a6-49e8-908e-807aa05d14e6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.351827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.352898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.353273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.353629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.355622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.356383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.375895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 799/1621] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:21.375895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.385827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 823/1622] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:21 2022] PUT /identity/v3/projects/9b95ed1c6d8c42778bb91e846e63d9d9/users/1a80343ac8154e70b013c93eaf7c380a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:21.386706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.424266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce8650-e8a2-4c9f-852f-4d84f1ec2200 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.495094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ba213-0d82-44da-8d82-d87801feb027 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.502366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.502997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.503624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.503889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.504609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.506867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.509476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.510150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.510426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.510697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.511474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.514327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.522846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 824/1623] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:21.526407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.538037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221f6c4-a840-43ce-8362-285c3ab60b32 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.543886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.544664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.545038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.545347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.547395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.547778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.558789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 825/1624] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:21.559510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.570457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa6230-3edc-411e-8cab-43aaa620fb94 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.575555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0d20b-6c34-4cbf-b4bc-a4674016c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.577076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d20b-6c34-4cbf-b4bc-a4674016c6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.577433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d20b-6c34-4cbf-b4bc-a4674016c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.577777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d20b-6c34-4cbf-b4bc-a4674016c6c8 admin admin] PATH_INFO: `/v3/projects/eda064e87e42404db4a124ff3a9edcff/users/bd144b47045847e5932378584ef9ca65/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.596741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0d20b-6c34-4cbf-b4bc-a4674016c6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda064e87e42404db4a124ff3a9edcff, user_id=bd144b47045847e5932378584ef9ca65, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.600626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0d20b-6c34-4cbf-b4bc-a4674016c6c8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.657651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 800/1625] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:21 2022] POST /identity/v3/users => generated 299 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:21.657651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.669481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04108a2a-1513-48d9-a21a-37d44ee4a7ec admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.675539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.676238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.676544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.676849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] PATH_INFO: `/v3/projects/9b95ed1c6d8c42778bb91e846e63d9d9/users/1a80343ac8154e70b013c93eaf7c380a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.679017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b95ed1c6d8c42778bb91e846e63d9d9, user_id=1a80343ac8154e70b013c93eaf7c380a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.681731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.693235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 801/1626] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:21 2022] PUT /identity/v3/projects/9b95ed1c6d8c42778bb91e846e63d9d9/users/1a80343ac8154e70b013c93eaf7c380a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:21.693851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.722880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 826/1627] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:21 2022] PUT /identity/v3/projects/eda064e87e42404db4a124ff3a9edcff/users/bd144b47045847e5932378584ef9ca65/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:21.724613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adee64a-a10c-4dfb-b578-4817b9370a57 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.725868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adee64a-a10c-4dfb-b578-4817b9370a57 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.726325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adee64a-a10c-4dfb-b578-4817b9370a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.760823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4adee64a-a10c-4dfb-b578-4817b9370a57 None None] MFA Rules not processed for user `1a80343ac8154e70b013c93eaf7c380a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:21.762322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4adee64a-a10c-4dfb-b578-4817b9370a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:21.792107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e190f3-4e6b-473c-aadc-159a74e78788 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.858598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 827/1628] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:21.860022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.860319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.860761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.871996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.874613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.875139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.875618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:21.876439 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:21.878347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:21.908100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 802/1629] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:21.909620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:21.912935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:21.923031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] MFA Rules not processed for user `bd144b47045847e5932378584ef9ca65`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:21.926612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:21.990316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801e42b-0c34-4c3e-8eae-5f0aa24d1d31 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:21.994247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:21.995152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:21.995698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:21.996093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.016592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 828/1630] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:22.018083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.033254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281db16-ba37-4c25-ae42-48c2a254af4b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.038301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.038981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.039260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.039538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] PATH_INFO: `/v3/projects/1cea15e26c4b42dbae9b8b267ec5f2f2/users/e0fc204517c5442c9f44235cf217076a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.059427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cea15e26c4b42dbae9b8b267ec5f2f2, user_id=e0fc204517c5442c9f44235cf217076a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.062474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.083606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.085165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.096254 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 803/1631] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:22.097153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.109438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5303a54-d5b5-4a59-9684-085bb3ecb7e7 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.115601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.116242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.120127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.120127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.120127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.120127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.148955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 829/1632] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:22 2022] PUT /identity/v3/projects/1cea15e26c4b42dbae9b8b267ec5f2f2/users/e0fc204517c5442c9f44235cf217076a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:22.153771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.242374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 804/1633] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:22 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:22.248131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.270718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e02e65-c28f-4760-b99a-90bfb2ed95bd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.340345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:22.403292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a21ad7-4e2e-48a7-8b97-97e39b3fdbab admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.412943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:22.420832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.420832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.420832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.420832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.421891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.423635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.438706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.440015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.440510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.441133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.442680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.468029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.478588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 830/1634] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:22.480290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.500511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef19fe-f1c2-41ee-9ed9-9017b43807b0 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.509295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae7b59-325e-4dba-82a0-37364db8ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.510552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae7b59-325e-4dba-82a0-37364db8ca5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.510941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae7b59-325e-4dba-82a0-37364db8ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.511311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae7b59-325e-4dba-82a0-37364db8ca5d admin admin] PATH_INFO: `/v3/projects/1cea15e26c4b42dbae9b8b267ec5f2f2/users/e0fc204517c5442c9f44235cf217076a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.515472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae7b59-325e-4dba-82a0-37364db8ca5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cea15e26c4b42dbae9b8b267ec5f2f2, user_id=e0fc204517c5442c9f44235cf217076a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.518464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae7b59-325e-4dba-82a0-37364db8ca5d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.549532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 831/1635] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:22 2022] PUT /identity/v3/projects/1cea15e26c4b42dbae9b8b267ec5f2f2/users/e0fc204517c5442c9f44235cf217076a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:22.552454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.553631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.554294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.576307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 805/1636] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:22 2022] POST /identity/v3/users => generated 297 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:22.579609 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.598180 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba0f40-4052-4c2c-baef-05a63cd820da admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.615500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] MFA Rules not processed for user `e0fc204517c5442c9f44235cf217076a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:22.616777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:22.732301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.733124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.733548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.733937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.734661 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.736389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.748874 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 832/1637] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:22.753782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 806/1638] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:22.755257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.774797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626cd41-8198-4167-a5e5-d9953e4e36c5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:22.782706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:22.783525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:22.784801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:22.784801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] PATH_INFO: `/v3/projects/be2f5bf0bd6a4a84b4c3c2065d1b4670/users/6678c1ca16e8450284dabc7387be4458/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:22.813245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2f5bf0bd6a4a84b4c3c2065d1b4670, user_id=6678c1ca16e8450284dabc7387be4458, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:22.818300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:22.904158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:22.909219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 833/1639] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:22 2022] PUT /identity/v3/projects/be2f5bf0bd6a4a84b4c3c2065d1b4670/users/6678c1ca16e8450284dabc7387be4458/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:22.930720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654fdf6-e919-40d7-87b9-ac470abd8583 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:23.008677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69405-2b39-4453-a7a4-2bed7db6f2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:23.008677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69405-2b39-4453-a7a4-2bed7db6f2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.013527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69405-2b39-4453-a7a4-2bed7db6f2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.013527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69405-2b39-4453-a7a4-2bed7db6f2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.013527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69405-2b39-4453-a7a4-2bed7db6f2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:23.013527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69405-2b39-4453-a7a4-2bed7db6f2a6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:23.025449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 807/1640] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:23.028333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:23.043175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df97f60-7563-4a26-9c17-c27d90181314 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:23.046258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:23.047078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.047362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.047635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] PATH_INFO: `/v3/projects/be2f5bf0bd6a4a84b4c3c2065d1b4670/users/6678c1ca16e8450284dabc7387be4458/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.049731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2f5bf0bd6a4a84b4c3c2065d1b4670, user_id=6678c1ca16e8450284dabc7387be4458, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:23.052501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:23.067551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 834/1641] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:23 2022] PUT /identity/v3/projects/be2f5bf0bd6a4a84b4c3c2065d1b4670/users/6678c1ca16e8450284dabc7387be4458/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:23.078632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.078947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.079184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.118370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] MFA Rules not processed for user `6678c1ca16e8450284dabc7387be4458`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:23.124741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:23.139415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:23.142720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:23.224293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 808/1642] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:23.242912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb62ca-5c23-4eb1-a0ce-c7d80423dc35 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:23.248750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:23.249728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.250007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.250285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.335467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:23.347660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474f1d0-f2f9-493b-aaed-f39dd60ac778 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:23.351597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:23.352225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.352870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.353204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.355120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:23.356573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:23.367506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:23.368959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:23.370893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 835/1643] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:23.387459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 809/1644] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:23.462489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:23.506992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:23.540412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0f65f-0e6f-46ec-841e-d5fcfa7e9fbe service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:23.546701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:23.547676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.548188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.548666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] PATH_INFO: `/v3/users/20d81ee13cb9482eaaa19ab797f1ad98` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.550548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d81ee13cb9482eaaa19ab797f1ad98)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:23.553962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:23.588271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:23.602276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1971547-65b4-480e-88d1-9fb2cb7f52ac service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:23.611127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a40e1-aa64-47e8-8846-d578627218a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:23.612455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a40e1-aa64-47e8-8846-d578627218a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.612892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a40e1-aa64-47e8-8846-d578627218a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.613370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a40e1-aa64-47e8-8846-d578627218a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.726767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a40e1-aa64-47e8-8846-d578627218a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:23.728471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a40e1-aa64-47e8-8846-d578627218a3 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:23.739389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 810/1645] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:23.800969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d81ee13cb9482eaaa19ab797f1ad98'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:23.830229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:23.847013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d81ee13cb9482eaaa19ab797f1ad98'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:23.866824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d81ee13cb9482eaaa19ab797f1ad98'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:23.867548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d81ee13cb9482eaaa19ab797f1ad98'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:23.868172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d81ee13cb9482eaaa19ab797f1ad98'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:23.880856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 836/1646] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:23 2022] DELETE /identity/v3/users/20d81ee13cb9482eaaa19ab797f1ad98 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:23.887119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:23.887457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:23.887688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:23.913578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:23.914823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:24.060878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:24.106117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 811/1647] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:24.123964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:24.126449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:24.208179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f7d2f-4173-4237-b26b-bed8cfe385c0 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:24.213112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ee512-ba58-4741-b8a2-b59520e80fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:24.213964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee512-ba58-4741-b8a2-b59520e80fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:24.214357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee512-ba58-4741-b8a2-b59520e80fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:24.214728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee512-ba58-4741-b8a2-b59520e80fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:24.227400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ee512-ba58-4741-b8a2-b59520e80fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:24.229107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ee512-ba58-4741-b8a2-b59520e80fe0 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:24.240077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 837/1648] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:24.276645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:24.280157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:24.294745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bda2bb-51b4-4027-b1eb-85fbcfedbf8d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:24.301643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:24.302596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:24.303017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:24.303436 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] PATH_INFO: `/v3/users/bd144b47045847e5932378584ef9ca65` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:24.305153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd144b47045847e5932378584ef9ca65)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:24.308153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:24.448278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:24.502253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:24.518182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd144b47045847e5932378584ef9ca65'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.531408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd144b47045847e5932378584ef9ca65'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.578874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd144b47045847e5932378584ef9ca65'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.579418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd144b47045847e5932378584ef9ca65'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.580231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd144b47045847e5932378584ef9ca65'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.600904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 812/1649] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:24 2022] DELETE /identity/v3/users/bd144b47045847e5932378584ef9ca65 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:24.605759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:24.606099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:24.606378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:24.626184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:24.627492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:24.688337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:24.691668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:24.693588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 838/1650] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:24.707346 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:24.709463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:24.721287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c3890-705c-4584-a98e-5be971db2800 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:24.726754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:24.728013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:24.728468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:24.728736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] PATH_INFO: `/v3/projects/5d7cbc53a96c40a99d32ab6b0c6d2f21` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:24.730055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7cbc53a96c40a99d32ab6b0c6d2f21)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:24.733169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:24.788973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d2150-4b7f-472f-a67e-1ad052aa7fc2 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:24.794023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:24.795565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:24.796023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:24.796480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:24.809467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:24.811499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:24.826093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 839/1651] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:24.868688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7cbc53a96c40a99d32ab6b0c6d2f21', 'request_id': 'req-ca905e79-0973-47df-b12d-c6c7c4f7848a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.876865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7cbc53a96c40a99d32ab6b0c6d2f21', 'request_id': 'req-ca905e79-0973-47df-b12d-c6c7c4f7848a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.883136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7cbc53a96c40a99d32ab6b0c6d2f21', 'request_id': 'req-ca905e79-0973-47df-b12d-c6c7c4f7848a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.896703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:24.922129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:24.943797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] The token cache is being invalidate because project 5d7cbc53a96c40a99d32ab6b0c6d2f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:24.944080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.945178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:24.946728 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 813/1652] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:24 2022] DELETE /identity/v3/projects/5d7cbc53a96c40a99d32ab6b0c6d2f21 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:24.951305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:24.961395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026f08b-cb28-4545-9cf4-eb96b2d1b309 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:25.020764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:25.021471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:25.021761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:25.022035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] PATH_INFO: `/v3/users/e0fc204517c5442c9f44235cf217076a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:25.023244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0fc204517c5442c9f44235cf217076a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:25.025199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:25.236278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0fc204517c5442c9f44235cf217076a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.239737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:25.241301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:25.256447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca905e79-0973-47df-b12d-c6c7c4f7848a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:25.267174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:25.267819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:25.268168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:25.268395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] PATH_INFO: `/v3/projects/eda064e87e42404db4a124ff3a9edcff` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:25.269595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda064e87e42404db4a124ff3a9edcff)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:25.271511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:25.322176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0fc204517c5442c9f44235cf217076a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.337917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0fc204517c5442c9f44235cf217076a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.338706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0fc204517c5442c9f44235cf217076a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.339311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0fc204517c5442c9f44235cf217076a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.354871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 840/1653] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:24 2022] DELETE /identity/v3/users/e0fc204517c5442c9f44235cf217076a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:25.371345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:25.374784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda064e87e42404db4a124ff3a9edcff', 'request_id': 'req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.381703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda064e87e42404db4a124ff3a9edcff', 'request_id': 'req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.387748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda064e87e42404db4a124ff3a9edcff', 'request_id': 'req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.464134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] The token cache is being invalidate because project eda064e87e42404db4a124ff3a9edcff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:25.464419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.465093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.466777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 814/1654] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:25 2022] DELETE /identity/v3/projects/eda064e87e42404db4a124ff3a9edcff => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:25.473098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:25.474863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:25.522558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:25.548813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6671657-8d57-4c23-acd7-0bec56cecd0f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:25.615317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:25.616001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:25.616285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:25.616562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] PATH_INFO: `/v3/users/6678c1ca16e8450284dabc7387be4458` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:25.617762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6678c1ca16e8450284dabc7387be4458)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:25.619796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:25.774791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:25.776260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:25.790696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab759-ad16-4488-91fd-4c8e77fa7af9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:25.796779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:25.797646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:25.798120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:25.798581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] PATH_INFO: `/v3/projects/1cea15e26c4b42dbae9b8b267ec5f2f2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:25.801170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cea15e26c4b42dbae9b8b267ec5f2f2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:25.805568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:25.865427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6678c1ca16e8450284dabc7387be4458'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.933531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cea15e26c4b42dbae9b8b267ec5f2f2', 'request_id': 'req-a56d922d-95fa-4a2b-9049-948b1c290c1e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.938785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6678c1ca16e8450284dabc7387be4458'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.941259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cea15e26c4b42dbae9b8b267ec5f2f2', 'request_id': 'req-a56d922d-95fa-4a2b-9049-948b1c290c1e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.946951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cea15e26c4b42dbae9b8b267ec5f2f2', 'request_id': 'req-a56d922d-95fa-4a2b-9049-948b1c290c1e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.952266 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6678c1ca16e8450284dabc7387be4458'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.952981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6678c1ca16e8450284dabc7387be4458'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.953614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ec9c7cc-db67-4789-9a3f-5e9aa84bd5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6678c1ca16e8450284dabc7387be4458'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:25.966586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 841/1655] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:25 2022] DELETE /identity/v3/users/6678c1ca16e8450284dabc7387be4458 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:26.048538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] The token cache is being invalidate because project 1cea15e26c4b42dbae9b8b267ec5f2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:26.048916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.049715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a56d922d-95fa-4a2b-9049-948b1c290c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.051357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 815/1656] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:25 2022] DELETE /identity/v3/projects/1cea15e26c4b42dbae9b8b267ec5f2f2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:26.056472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.056928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.057251 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.079701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:26.080916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.121577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:26.146012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 842/1657] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:26.154565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.154896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.155170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.176531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:26.177840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.186256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 816/1658] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:26.190759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.201697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540032e6-1d05-4a85-9051-67f31d825a16 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.207561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.208184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.208486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.208715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.209441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:26.209684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.211304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.226301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 843/1659] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:26.230912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.231158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:26.241346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60425458-0a61-4b8d-9dd9-1f70d7fda4f3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.249999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.250850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.251137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.251505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.252143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.254198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.335454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 817/1660] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:26.339627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.350163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6ae09-64d8-4a30-bed5-c29b0d4495cd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.383643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.386643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.463949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.464726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.465017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.465351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.466561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.470220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.507240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6282-7bf1-4988-9847-abea14da8b17 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.519674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.520347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.520622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.520888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] PATH_INFO: `/v3/projects/be2f5bf0bd6a4a84b4c3c2065d1b4670` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.522422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2f5bf0bd6a4a84b4c3c2065d1b4670)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.524487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.557679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:26.568216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 844/1661] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:26.572202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.583001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c23f9-680d-4dac-a34b-4014c7efe565 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.587630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.588229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.588511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.588769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.589435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.591729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.604575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 845/1662] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:26.608548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.618779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69b5ba-5b01-4cfc-b18b-3d6a96bb8f49 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.623646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.624302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.624549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.624817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.625516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.627130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.637497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 846/1663] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:26.642206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.652053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919274c-83e0-4b26-be6e-e8b021d1f78b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.657442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611f295-12cd-49c5-bde9-f93deb89e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.658123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611f295-12cd-49c5-bde9-f93deb89e2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.658403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611f295-12cd-49c5-bde9-f93deb89e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.658826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611f295-12cd-49c5-bde9-f93deb89e2b0 admin admin] PATH_INFO: `/v3/projects/ebcdaf586a0640edbede169f3c3c3277/users/725b2b4c29e440cda491336a097e9903/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.674648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611f295-12cd-49c5-bde9-f93deb89e2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcdaf586a0640edbede169f3c3c3277, user_id=725b2b4c29e440cda491336a097e9903, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.677365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611f295-12cd-49c5-bde9-f93deb89e2b0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.677653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2f5bf0bd6a4a84b4c3c2065d1b4670', 'request_id': 'req-4bccc070-9242-4c2f-9801-8f58f5aca1cf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.684169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2f5bf0bd6a4a84b4c3c2065d1b4670', 'request_id': 'req-4bccc070-9242-4c2f-9801-8f58f5aca1cf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.689871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2f5bf0bd6a4a84b4c3c2065d1b4670', 'request_id': 'req-4bccc070-9242-4c2f-9801-8f58f5aca1cf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.761759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] The token cache is being invalidate because project be2f5bf0bd6a4a84b4c3c2065d1b4670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:26.762372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.762594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 847/1664] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:26 2022] PUT /identity/v3/projects/ebcdaf586a0640edbede169f3c3c3277/users/725b2b4c29e440cda491336a097e9903/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:26.769473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4bccc070-9242-4c2f-9801-8f58f5aca1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:26.771064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 818/1665] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:26 2022] DELETE /identity/v3/projects/be2f5bf0bd6a4a84b4c3c2065d1b4670 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:26.773613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.774047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.774329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.776925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fc908-16e6-4f1d-9f14-1baf3e3be239 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.777422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fc908-16e6-4f1d-9f14-1baf3e3be239 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.777653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fc908-16e6-4f1d-9f14-1baf3e3be239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.799206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-506fc908-16e6-4f1d-9f14-1baf3e3be239 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:26.800557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-506fc908-16e6-4f1d-9f14-1baf3e3be239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.806572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] MFA Rules not processed for user `725b2b4c29e440cda491336a097e9903`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:26.807892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.823527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:26.875889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 819/1666] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:26.884248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.884572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.885196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.909127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 848/1667] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:26.912938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.917424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.917974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:26.919349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:26.928036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:26.929148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 820/1668] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:26.932990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.935693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e2f22-cca3-4e83-a95f-84a9b3224e71 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.944107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2c4c5-ad9a-48e6-8f84-13a3ccc330e7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.946408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.947038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.947354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.947570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.948205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.951701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.952117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.952840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.953157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.953426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.954081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:26.954497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.956258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:26.968535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 821/1669] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:40:26.973196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:26.983386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf9b9-8147-4091-8445-8b3c42f8779f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:26.988675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:26.989303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:26.989580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:26.989977 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:26.990625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:26.992489 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.094914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 849/1670] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/projects => generated 393 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:27.098998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.124699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 822/1671] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:26 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:27.129445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.164817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5e8bf-ce93-46a6-a7e3-de9f6aba440a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.195069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29422018-0598-49d4-8427-8e119e9380fd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.200104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.200788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.201261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.201535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.202211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.204243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.226381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.227004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.227344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.227871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.228512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.230587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.300110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 823/1672] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:27 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:27.304206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.314947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74247a6-9b2f-43cd-9fcb-21788ad7ad76 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.317483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:27.321665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.322278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.322559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.322670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 850/1673] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:27 2022] POST /identity/v3/users => generated 303 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:27.322824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.323471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.325261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.326675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.335961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 824/1674] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:27.336946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e45029-6d1c-4009-b952-e6156d9bb1e8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.339971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.341868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.342478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.342758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.343029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.343735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.353547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.353686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b58d4-da17-4ae1-adc9-bb48535e7328 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.355798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.356403 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.357817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.357955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 851/1675] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:27.358137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.358786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.360387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.362384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.371808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 825/1676] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:27.373227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9b078-7a2b-4229-9381-c7551615e33e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.376702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.381972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.381972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.381972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.381972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] PATH_INFO: `/v3/projects/ab15071419e840388cbb160827e66424/users/d1dcb425552340fead743d378d9775fc/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.386958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491ae33-4d54-410d-9615-15db7043cae5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.391985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a41f1-0c14-416e-b06e-6d12477f385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.392710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a41f1-0c14-416e-b06e-6d12477f385f admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.393080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a41f1-0c14-416e-b06e-6d12477f385f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.393300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a41f1-0c14-416e-b06e-6d12477f385f admin admin] PATH_INFO: `/v3/projects/25f0bbc5c14b4fc496443813be821dfc/users/b94f3cd454634df486808360afa3de83/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.397691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab15071419e840388cbb160827e66424, user_id=d1dcb425552340fead743d378d9775fc, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.400406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.408529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a41f1-0c14-416e-b06e-6d12477f385f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f0bbc5c14b4fc496443813be821dfc, user_id=b94f3cd454634df486808360afa3de83, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.411719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a41f1-0c14-416e-b06e-6d12477f385f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.453245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:27.470172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 852/1677] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:27 2022] PUT /identity/v3/projects/ab15071419e840388cbb160827e66424/users/d1dcb425552340fead743d378d9775fc/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:27.474233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.529109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 826/1678] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:27 2022] PUT /identity/v3/projects/25f0bbc5c14b4fc496443813be821dfc/users/b94f3cd454634df486808360afa3de83/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:27.535719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.536189 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.536638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.560030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272f82a-cf3e-43fe-aded-c48c2dea8a27 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.610846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] MFA Rules not processed for user `b94f3cd454634df486808360afa3de83`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:27.613654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:27.650698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:27.671069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.673157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.674083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.674706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.676699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.681878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.704847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 853/1679] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:27.705836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.732328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0f3c-cd91-4011-994a-487ceaab9671 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.740425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee71e55-28b1-4c57-87dd-f23646932ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.742190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee71e55-28b1-4c57-87dd-f23646932ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.742739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee71e55-28b1-4c57-87dd-f23646932ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.743205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee71e55-28b1-4c57-87dd-f23646932ffe admin admin] PATH_INFO: `/v3/projects/ab15071419e840388cbb160827e66424/users/d1dcb425552340fead743d378d9775fc/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.750898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee71e55-28b1-4c57-87dd-f23646932ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab15071419e840388cbb160827e66424, user_id=d1dcb425552340fead743d378d9775fc, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:27.753457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee71e55-28b1-4c57-87dd-f23646932ffe admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:27.763400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:27.785106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 854/1680] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:27 2022] PUT /identity/v3/projects/ab15071419e840388cbb160827e66424/users/d1dcb425552340fead743d378d9775fc/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:27.793782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.794743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.794916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:27.799154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 827/1681] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:27.811743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:27.813378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:27.860563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:27.860667 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] MFA Rules not processed for user `d1dcb425552340fead743d378d9775fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:27.861668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:27.927661 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717db9d4-71a1-46cc-b796-697361bfd6d2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:27.961100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 855/1682] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:27.978868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:27.979480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:27.979755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:27.980022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:28.036418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:28.037881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:28.047023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 828/1683] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:28.202717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:28.202717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:28.258497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ebdc2-c8dd-460f-b3a9-fb42e9c6e78e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:28.263190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:28.264092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:28.264375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:28.264646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:28.274656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:28.276096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:28.287950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 856/1684] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:28.292505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:28.453616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:28.513637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:28.526245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbb776-081b-4793-afb1-786700ef9772 service glance] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:28.530588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:28.531213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:28.531503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:28.531796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:28.591555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:28.593194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:28.602379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 829/1685] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:29.197246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:29.362370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:30.345095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:30.383087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:31.151791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=7 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:31.293231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:31.693583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:31.750963 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:31.941429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:32.219063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:32.222070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:32.232446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6121c0f-4861-4b22-acf6-89d2862476d4 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:32.236126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:32.236721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:32.237023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:32.237299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:32.238992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:32.307587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:32.308941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737060a-98e7-4723-9a1a-bbaaf402bb53 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:32.319494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 857/1686] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:32.360230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:32.516228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:32.859189 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:33.093872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:33.435247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:33.493113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:33.834445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:33.837251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:33.872319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:33.931064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916c8c-9e94-4a7f-98c3-8e18ae779bdc service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:33.936773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:33.944098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:33.944098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:33.944098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] PATH_INFO: `/v3/users/c45cd5543178463f85a7c7f062389a5f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:33.944098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45cd5543178463f85a7c7f062389a5f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:33.944098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:34.180398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45cd5543178463f85a7c7f062389a5f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:34.191614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:34.196962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45cd5543178463f85a7c7f062389a5f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:34.282239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:34.307099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45cd5543178463f85a7c7f062389a5f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:34.307901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45cd5543178463f85a7c7f062389a5f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:34.308657 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45cd5543178463f85a7c7f062389a5f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:34.327482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 830/1687] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:33 2022] DELETE /identity/v3/users/c45cd5543178463f85a7c7f062389a5f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:34.333385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:34.334175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:34.334685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:34.369768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:34.369768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:34.435391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 858/1688] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:34.445915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:34.447343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:34.512322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a184-2898-4ed6-b1a7-e604813693b7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:34.517698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:34.518586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:34.518703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:34.518934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:34.529844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:34.531275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:34.542550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 831/1689] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:34.751204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:34.917486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:35.077932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:35.080498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:35.105991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16950216-6fed-4f9c-9dab-45f5f77a9ac4 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:35.116510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:35.118647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:35.118647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:35.118647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] PATH_INFO: `/v3/projects/215b01dc4fa14b1da39498761f0018de` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:35.119913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=215b01dc4fa14b1da39498761f0018de)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:35.122020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:35.264338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '215b01dc4fa14b1da39498761f0018de', 'request_id': 'req-43c731dc-291f-40c8-89dc-03aa04982c81'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.271239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '215b01dc4fa14b1da39498761f0018de', 'request_id': 'req-43c731dc-291f-40c8-89dc-03aa04982c81'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.333857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '215b01dc4fa14b1da39498761f0018de', 'request_id': 'req-43c731dc-291f-40c8-89dc-03aa04982c81'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.346784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] The token cache is being invalidate because project 215b01dc4fa14b1da39498761f0018de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:35.347082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.347799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.349360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 859/1690] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:35 2022] DELETE /identity/v3/projects/215b01dc4fa14b1da39498761f0018de => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:35.353993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:35.364789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4c18f-8b8a-4c59-916a-aa37971b65bc service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:35.444369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:35.445573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:35.446540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:35.446682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] PATH_INFO: `/v3/users/34f40786d942450ea55c5f35c3f85fb6` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:35.448762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f40786d942450ea55c5f35c3f85fb6)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:35.452243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:35.500026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:35.618659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:35.715283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f40786d942450ea55c5f35c3f85fb6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.729725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f40786d942450ea55c5f35c3f85fb6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.826046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f40786d942450ea55c5f35c3f85fb6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.826851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f40786d942450ea55c5f35c3f85fb6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.827516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f40786d942450ea55c5f35c3f85fb6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:35.840987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 832/1691] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:35 2022] DELETE /identity/v3/users/34f40786d942450ea55c5f35c3f85fb6 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:36.015065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:36.141420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:36.143014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.157157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c731dc-291f-40c8-89dc-03aa04982c81 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:36.248485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:36.264123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:36.264826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:36.265422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:36.266429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] PATH_INFO: `/v3/users/725b2b4c29e440cda491336a097e9903` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:36.268385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725b2b4c29e440cda491336a097e9903)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:36.270499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:36.478028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:36.479473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.493086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e493c9e-7318-4d3a-88e0-556d424a3c64 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:36.498767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:36.501494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:36.501494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:36.501494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] PATH_INFO: `/v3/projects/ff21a7c295b44bb29782029696e1b7bd` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:36.501494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff21a7c295b44bb29782029696e1b7bd)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:36.503511 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:36.589172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725b2b4c29e440cda491336a097e9903'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.631227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff21a7c295b44bb29782029696e1b7bd', 'request_id': 'req-2f9bb219-554a-45b4-8bf2-f5577a202c4e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.639975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff21a7c295b44bb29782029696e1b7bd', 'request_id': 'req-2f9bb219-554a-45b4-8bf2-f5577a202c4e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.646554 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff21a7c295b44bb29782029696e1b7bd', 'request_id': 'req-2f9bb219-554a-45b4-8bf2-f5577a202c4e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.728258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725b2b4c29e440cda491336a097e9903'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.741758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725b2b4c29e440cda491336a097e9903'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.742473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725b2b4c29e440cda491336a097e9903'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.743086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-621d5dab-9056-4b7e-a91d-25cccb7dd665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725b2b4c29e440cda491336a097e9903'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.766520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 860/1692] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:36 2022] DELETE /identity/v3/users/725b2b4c29e440cda491336a097e9903 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:36.770883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:36.771237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:36.771594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:36.790750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:36.791942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.854977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 861/1693] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:36.864964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:36.866549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.873600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:36.876438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] The token cache is being invalidate because project ff21a7c295b44bb29782029696e1b7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:36.876737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.877411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2f9bb219-554a-45b4-8bf2-f5577a202c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:36.879119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 833/1694] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:36 2022] DELETE /identity/v3/projects/ff21a7c295b44bb29782029696e1b7bd => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:36.883909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cefe9e-d872-48d5-a253-581b87523cc1 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:36.884196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cefe9e-d872-48d5-a253-581b87523cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:36.884485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cefe9e-d872-48d5-a253-581b87523cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:36.916591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-12cefe9e-d872-48d5-a253-581b87523cc1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:36.917846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-12cefe9e-d872-48d5-a253-581b87523cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.927172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 834/1695] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:36.936694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:36.938367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:36.938367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:36.959289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaee51b-0a16-426c-846c-4a0f4ae2fbbf None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:36.963553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:36.964593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:36.964886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:36.965106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:36.972099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.988540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:36.988540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:36.991792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:36.993941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.022015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 835/1697] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:37.029646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 862/1697] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:37.038291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.049153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a961942-4696-4c62-b6fa-f9b522516638 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.054063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.054814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.055213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.055545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.056464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:37.056816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.059049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.072764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 836/1698] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:37.077748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.088349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dae69-ac33-4b2d-9338-bddf97ea78da service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.093474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.094095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.094378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.094653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.095285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.097353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.215186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 863/1699] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:37 2022] POST /identity/v3/projects => generated 363 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:37.220441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.232991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c80ba-5119-4b91-9b17-f8c3c2177e83 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.310871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:37.320783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.321739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.322172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.322650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.323550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.326303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.425349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:37.432129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 837/1700] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:37 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:37.436767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.447344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b787d3-77fc-426e-938d-72f2c1bc5ce2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.452038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.452756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.453075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.453403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.454112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.455721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.466793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 864/1701] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:37.470882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.482566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a4621-8b84-4a64-b2e9-f4a8cdc509b5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.487663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.488283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.488562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.488831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.489504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.491146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.502620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 838/1702] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:37.507279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.518715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccfbe7-2254-4096-91ed-bf7c6420e5dd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.522723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b142dd0-4350-4a15-b35d-ef1e26fa58e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.523403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142dd0-4350-4a15-b35d-ef1e26fa58e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.523686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142dd0-4350-4a15-b35d-ef1e26fa58e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.523958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142dd0-4350-4a15-b35d-ef1e26fa58e0 admin admin] PATH_INFO: `/v3/projects/1911094c2ace4c29ace6678190932d46/users/4a9bdc4925c44fc7946fc3877c85993c/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.540459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b142dd0-4350-4a15-b35d-ef1e26fa58e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1911094c2ace4c29ace6678190932d46, user_id=4a9bdc4925c44fc7946fc3877c85993c, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.543271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b142dd0-4350-4a15-b35d-ef1e26fa58e0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.592802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.594707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:37.610379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8ad56-0b1d-4aeb-b403-8a095f1d66e0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.615887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.617117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.617403 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.617683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] PATH_INFO: `/v3/projects/ebcdaf586a0640edbede169f3c3c3277` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.619062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcdaf586a0640edbede169f3c3c3277)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.621917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.656600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 865/1703] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:37 2022] PUT /identity/v3/projects/1911094c2ace4c29ace6678190932d46/users/4a9bdc4925c44fc7946fc3877c85993c/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:37.668801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.669648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.669939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.703293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] MFA Rules not processed for user `4a9bdc4925c44fc7946fc3877c85993c`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:37.704519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:37.776985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 866/1704] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:37.782197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.806972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcdaf586a0640edbede169f3c3c3277', 'request_id': 'req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:37.816349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcdaf586a0640edbede169f3c3c3277', 'request_id': 'req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:37.823288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcdaf586a0640edbede169f3c3c3277', 'request_id': 'req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:37.860669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467d30d-13a6-4858-a544-88bd5b4e2095 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:37.910691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] The token cache is being invalidate because project ebcdaf586a0640edbede169f3c3c3277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:37.910985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:37.911697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:37.913298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 839/1705] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:37 2022] DELETE /identity/v3/projects/ebcdaf586a0640edbede169f3c3c3277 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:37.917313 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:37.919326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:37.930932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:37.931561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:37.931833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:37.932371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:37.932936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:37.935147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:37.998564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fc2f2-19ed-4d97-8ae2-43ba5e4233b5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.003449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.004113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.004388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.009275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] PATH_INFO: `/v3/users/d1dcb425552340fead743d378d9775fc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.009275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dcb425552340fead743d378d9775fc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.009275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.048071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 867/1706] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:37 2022] POST /identity/v3/projects => generated 365 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:38.048992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.050762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:38.064488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:38.145391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646d08-4d7d-4787-87c7-9ddac86691ed admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.150505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.151127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.151405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.151676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.152310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.155225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.240731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dcb425552340fead743d378d9775fc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:38.251059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 868/1707] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:38 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:38.252272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.256879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dcb425552340fead743d378d9775fc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:38.263989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfdbf0-dbf0-449b-a0bd-21a183ab29c1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.269388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.271527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.271527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.271527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.271527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.273061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.284837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 869/1708] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:38.289136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.301671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5980428-0b7a-49d0-bdae-88f51796fa02 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.304451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.304814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.305126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.305400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] PATH_INFO: `/v3/projects/11141ef34ba34be1ade403778fefdbe5/users/80728cb0f4de42f0b5188698784cbafb/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.321946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=11141ef34ba34be1ade403778fefdbe5, user_id=80728cb0f4de42f0b5188698784cbafb, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.324592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.339043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dcb425552340fead743d378d9775fc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:38.339671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dcb425552340fead743d378d9775fc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:38.341522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dcb425552340fead743d378d9775fc'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:38.360720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 840/1709] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:37 2022] DELETE /identity/v3/users/d1dcb425552340fead743d378d9775fc => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:38.428601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 870/1710] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:38 2022] PUT /identity/v3/projects/11141ef34ba34be1ade403778fefdbe5/users/80728cb0f4de42f0b5188698784cbafb/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:38.432448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.434157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:38.452094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:38.536600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7920c-225f-453f-baca-c95f95ed2634 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.568288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:38.600228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.601030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.601420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.601774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.602530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.604649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.617050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 841/1711] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:38.624353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.640837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a20fc8-276f-40d3-b020-2ea1f490cedc admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.655656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10273a2f-c13a-48a7-ae83-7205ca1912d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.656438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10273a2f-c13a-48a7-ae83-7205ca1912d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.658323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10273a2f-c13a-48a7-ae83-7205ca1912d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.658323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10273a2f-c13a-48a7-ae83-7205ca1912d4 admin admin] PATH_INFO: `/v3/projects/11141ef34ba34be1ade403778fefdbe5/users/80728cb0f4de42f0b5188698784cbafb/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.669768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10273a2f-c13a-48a7-ae83-7205ca1912d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11141ef34ba34be1ade403778fefdbe5, user_id=80728cb0f4de42f0b5188698784cbafb, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.673411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10273a2f-c13a-48a7-ae83-7205ca1912d4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.691649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 871/1712] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:38 2022] PUT /identity/v3/projects/11141ef34ba34be1ade403778fefdbe5/users/80728cb0f4de42f0b5188698784cbafb/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:38.710998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.711207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.711294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.711730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.713010 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:38.729226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d9c0c-41e3-4406-8419-ab9844e7ecf0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.746060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] MFA Rules not processed for user `80728cb0f4de42f0b5188698784cbafb`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:38.746060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:38.821326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.822392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.822675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.823238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] PATH_INFO: `/v3/users/e509668e9328439296f7dd64c563a81f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.824465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e509668e9328439296f7dd64c563a81f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.829387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.831469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 872/1713] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:38.844553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.845994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:38.922358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b7d6a-1cab-4dba-b655-62c89f5a6268 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.927957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.928716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.929191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.929704 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.940683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.942372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:38.953548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 873/1714] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:38.954519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:38.956008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:38.974233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ddf6e-3bd7-474f-8de4-8b10ea735ab2 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:38.979738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:38.980380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:38.980660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:38.981029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] PATH_INFO: `/v3/projects/ab15071419e840388cbb160827e66424` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:38.982292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab15071419e840388cbb160827e66424)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:38.984496 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.069614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e509668e9328439296f7dd64c563a81f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.086218 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e509668e9328439296f7dd64c563a81f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.100280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab15071419e840388cbb160827e66424', 'request_id': 'req-17a98f3d-5801-49ee-8c73-f55181f18cfd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.109853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab15071419e840388cbb160827e66424', 'request_id': 'req-17a98f3d-5801-49ee-8c73-f55181f18cfd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.172586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e509668e9328439296f7dd64c563a81f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.173478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e509668e9328439296f7dd64c563a81f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.180177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e571d407-78d2-470f-8f01-14bdb6d1a98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e509668e9328439296f7dd64c563a81f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.188425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 842/1715] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:38 2022] DELETE /identity/v3/users/e509668e9328439296f7dd64c563a81f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:39.192627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001dd35-e45c-4d41-af1d-4dd61c5398dc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.192904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001dd35-e45c-4d41-af1d-4dd61c5398dc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.193228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001dd35-e45c-4d41-af1d-4dd61c5398dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.212422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c001dd35-e45c-4d41-af1d-4dd61c5398dc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:39.213636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c001dd35-e45c-4d41-af1d-4dd61c5398dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.246739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab15071419e840388cbb160827e66424', 'request_id': 'req-17a98f3d-5801-49ee-8c73-f55181f18cfd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.259256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] The token cache is being invalidate because project ab15071419e840388cbb160827e66424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:39.259560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.265065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:39.265989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 874/1716] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:38 2022] DELETE /identity/v3/projects/ab15071419e840388cbb160827e66424 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:39.267125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.269718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.280567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 843/1717] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:39.282074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.282352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.282623 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.300066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:39.303849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:39.312060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.314159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 844/1718] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:39.314974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.316317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.331424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a98f3d-5801-49ee-8c73-f55181f18cfd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.333649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:39.376004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34dc29a-6108-4ab4-a717-bdb1468c9e6f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.379955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61339f7d-31e7-4998-9c4f-ffcb7906dcad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.380578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339f7d-31e7-4998-9c4f-ffcb7906dcad service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.380856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339f7d-31e7-4998-9c4f-ffcb7906dcad service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.381172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339f7d-31e7-4998-9c4f-ffcb7906dcad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.387096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.387714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.388848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.389160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.390977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.391415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61339f7d-31e7-4998-9c4f-ffcb7906dcad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.392940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61339f7d-31e7-4998-9c4f-ffcb7906dcad service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.402833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 845/1719] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:39.404249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.404508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.404834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.428049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:39.429346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.441863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 846/1720] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:39.443228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.453845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8ba2-75fb-4994-938b-7f78b7d38e33 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.459260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.459896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.460176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.460450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.461304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:39.461632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.463686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.467698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.468379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.478591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 847/1721] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:40:39.480373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 875/1722] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:39.483645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.495536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86eafd-5245-46b0-bd8e-2252144ffbc5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.503134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.503749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.504249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.504689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.505411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.507669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.593868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 848/1723] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:39 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:39.598075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.609792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0fd5b-270f-4610-9a6b-f388b0ecfe4a service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.655173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:39.674564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.676962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:39.677938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.678909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.678909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.679180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.683559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.769830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 876/1724] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:39 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:39.774006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.784940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920943b8-1d71-4c22-b3d9-7d051697a387 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.790266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.790866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.791143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.791412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.792052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.793800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.803999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 849/1725] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:39.808191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.818542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab200f-9ff5-419f-9c89-c7b929dd2240 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.823590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.824222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.824537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.824766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.825797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.827485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.836105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.837833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:39.849053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 877/1726] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:39.852049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:39.855243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:39.860692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98538ecb-b24c-4b20-91b1-aad8abb6354a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.867472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.868156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.868344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.868772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] PATH_INFO: `/v3/projects/7d90eea69e914f039443078ac05d0518` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.870271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d90eea69e914f039443078ac05d0518)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.873127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:39.881030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f79ca2-5272-458c-8db7-5f832bccf9d4 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:39.892658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ba49a-1d42-4697-9328-8a9f60174b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:39.893474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ba49a-1d42-4697-9328-8a9f60174b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:39.893836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ba49a-1d42-4697-9328-8a9f60174b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:39.894175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ba49a-1d42-4697-9328-8a9f60174b25 admin admin] PATH_INFO: `/v3/projects/53dec18a0324410ba87e9565719299e3/users/72738646b9a84e9ca9298c874f669b55/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:39.929873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ba49a-1d42-4697-9328-8a9f60174b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53dec18a0324410ba87e9565719299e3, user_id=72738646b9a84e9ca9298c874f669b55, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:39.933060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ba49a-1d42-4697-9328-8a9f60174b25 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.042333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:40.049430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 878/1727] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:39 2022] PUT /identity/v3/projects/53dec18a0324410ba87e9565719299e3/users/72738646b9a84e9ca9298c874f669b55/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:40.058216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d90eea69e914f039443078ac05d0518', 'request_id': 'req-8f5925cf-a95c-484b-8a08-ca79416ae752'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.060989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.061370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.061731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.067523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d90eea69e914f039443078ac05d0518', 'request_id': 'req-8f5925cf-a95c-484b-8a08-ca79416ae752'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.073170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d90eea69e914f039443078ac05d0518', 'request_id': 'req-8f5925cf-a95c-484b-8a08-ca79416ae752'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.091002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] MFA Rules not processed for user `72738646b9a84e9ca9298c874f669b55`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:40.093084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:40.149803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] The token cache is being invalidate because project 7d90eea69e914f039443078ac05d0518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:40.150084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.150725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.152466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 850/1728] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:39 2022] DELETE /identity/v3/projects/7d90eea69e914f039443078ac05d0518 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:40.156264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.157647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:40.161112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 879/1729] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:40.170442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.216106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:40.261462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5925cf-a95c-484b-8a08-ca79416ae752 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.263368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afb0d2-7b64-4778-bee4-9b7b9af505aa None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.267192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.267849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.268120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.268392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] PATH_INFO: `/v3/users/1a80343ac8154e70b013c93eaf7c380a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.271637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a80343ac8154e70b013c93eaf7c380a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.271749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.271749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.271926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.272036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.272036 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.272582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.274535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.287757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:40.370631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 880/1730] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:40 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:40.374639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.376259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:40.440159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b0b8-5abf-4caa-ad99-cc9e11707218 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.445074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.445697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.445961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.446232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.446857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.448762 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.484060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a80343ac8154e70b013c93eaf7c380a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.496350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a80343ac8154e70b013c93eaf7c380a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.520521 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 881/1731] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:40 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:40.524523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.535059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d269b7-35fc-4b31-9467-236bef464e07 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.539641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.540212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.540486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.540754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.541406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.543182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.543545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:40.553808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 882/1732] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:40.557836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.567970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f61f1-196a-4823-be0b-5ee9d4ef9d93 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.572632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.573311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.573590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.573867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] PATH_INFO: `/v3/projects/0a85905e33c24df38fe52fe77916153c/users/329a994ae0ab4632a4dc6eb84af70c0e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.583870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a80343ac8154e70b013c93eaf7c380a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.586697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a80343ac8154e70b013c93eaf7c380a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.586697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a80343ac8154e70b013c93eaf7c380a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:40.589751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a85905e33c24df38fe52fe77916153c, user_id=329a994ae0ab4632a4dc6eb84af70c0e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.593969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.600190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 851/1733] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:40 2022] DELETE /identity/v3/users/1a80343ac8154e70b013c93eaf7c380a => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:40.705459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:40.711260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 883/1734] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:40 2022] PUT /identity/v3/projects/0a85905e33c24df38fe52fe77916153c/users/329a994ae0ab4632a4dc6eb84af70c0e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:40.715354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.717354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:40.783465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028792b-1002-40a3-98d6-c2c6c2623514 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.844644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259556c-5137-4474-b988-2621b3deef70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.845329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259556c-5137-4474-b988-2621b3deef70 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.845611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259556c-5137-4474-b988-2621b3deef70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.845913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259556c-5137-4474-b988-2621b3deef70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.846564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259556c-5137-4474-b988-2621b3deef70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.848248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259556c-5137-4474-b988-2621b3deef70 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.859625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 852/1735] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:40.863792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:40.874982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d11eec-aa86-46ed-8b17-d6953688ff6b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:40.880181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:40.881059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.881345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.881618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] PATH_INFO: `/v3/projects/0a85905e33c24df38fe52fe77916153c/users/329a994ae0ab4632a4dc6eb84af70c0e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.883541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a85905e33c24df38fe52fe77916153c, user_id=329a994ae0ab4632a4dc6eb84af70c0e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:40.886326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:40.897978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 884/1736] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:40 2022] PUT /identity/v3/projects/0a85905e33c24df38fe52fe77916153c/users/329a994ae0ab4632a4dc6eb84af70c0e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:40.909637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:40.910665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:40.911386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:40.945659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] MFA Rules not processed for user `329a994ae0ab4632a4dc6eb84af70c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:40.947014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:40.953833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:41.005483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.007213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:41.032320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 853/1737] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:41.042404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.044641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:41.086728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ed1e5-a6a0-4ff5-83e8-718eee661c94 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.092149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.092790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.093095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.093373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] PATH_INFO: `/v3/projects/9b95ed1c6d8c42778bb91e846e63d9d9` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.094530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b95ed1c6d8c42778bb91e846e63d9d9)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.100370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.103815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6e8b-d642-4e10-8b5c-a3bf355f2c10 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.108009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a6a3e-f8ac-49d2-8c83-382db0109751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.108639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a6a3e-f8ac-49d2-8c83-382db0109751 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.109037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a6a3e-f8ac-49d2-8c83-382db0109751 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.109201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a6a3e-f8ac-49d2-8c83-382db0109751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.118605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a6a3e-f8ac-49d2-8c83-382db0109751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.120030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a6a3e-f8ac-49d2-8c83-382db0109751 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.131088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 854/1738] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:41.184002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b95ed1c6d8c42778bb91e846e63d9d9', 'request_id': 'req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:41.190822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b95ed1c6d8c42778bb91e846e63d9d9', 'request_id': 'req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:41.277343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b95ed1c6d8c42778bb91e846e63d9d9', 'request_id': 'req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:41.291309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] The token cache is being invalidate because project 9b95ed1c6d8c42778bb91e846e63d9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:41.291640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:41.292681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c6a22c4-3035-42c6-af81-0102ebcfbd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:41.294629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 885/1739] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:41 2022] DELETE /identity/v3/projects/9b95ed1c6d8c42778bb91e846e63d9d9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:41.300938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.301470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.301817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.333515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:41.334716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:41.401478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 855/1740] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:41.409709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.410039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.410361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.456401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:41.457090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:41.474349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 886/1741] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:41.479195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.495136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5baf7-9c2e-4244-aced-d5fa63128ebd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.501979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.501979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.501979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.501979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.502411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:41.502507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.504162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.518670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 856/1742] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:41.522925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.534363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5faea7-bd90-47b0-b173-e294a472190f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.544834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.545988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.546394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.546849 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.547863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.550167 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.636979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 887/1743] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:41 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:41.641538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.651826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8d6c7-cb3b-41ae-bb2e-698dfb53e064 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.683506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:41.691464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.693294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:41.713883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.714485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.714758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.715028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.716223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.718223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.744890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:41.762092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8b22f-1348-4790-9164-432a401735f2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.767480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.768109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.768382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.768649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] PATH_INFO: `/v3/users/72738646b9a84e9ca9298c874f669b55` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.769811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72738646b9a84e9ca9298c874f669b55)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.771713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.807619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 857/1744] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:41 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:41.812518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.822430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c22c-25cf-4de4-9c37-8ac70cd656b9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.828194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.829283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.829563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.829828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.830475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.832040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.842273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 858/1745] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:41.846164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.856119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f908f-144b-429e-8d36-9aecf9904646 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.860680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.861282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.861558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.861826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.862488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.864024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.875853 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 859/1746] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:41.879875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:41.890678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195747a-67b8-4a43-8d17-64df8340cea4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:41.895622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6aed3-5e22-4284-a9ad-920a16cc248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:41.896281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6aed3-5e22-4284-a9ad-920a16cc248c admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:41.896556 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6aed3-5e22-4284-a9ad-920a16cc248c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:41.896826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6aed3-5e22-4284-a9ad-920a16cc248c admin admin] PATH_INFO: `/v3/projects/4300bdb9630f4101a602fd6e36982fb1/users/438997bb70d94306a15b08f6841f1bc4/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:41.913430 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6aed3-5e22-4284-a9ad-920a16cc248c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4300bdb9630f4101a602fd6e36982fb1, user_id=438997bb70d94306a15b08f6841f1bc4, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:41.916081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6aed3-5e22-4284-a9ad-920a16cc248c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:41.974216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72738646b9a84e9ca9298c874f669b55'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:41.996434 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 860/1747] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:41 2022] PUT /identity/v3/projects/4300bdb9630f4101a602fd6e36982fb1/users/438997bb70d94306a15b08f6841f1bc4/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:42.010224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:42.010509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:42.010776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:42.060096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] MFA Rules not processed for user `438997bb70d94306a15b08f6841f1bc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:42.061582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:42.070716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72738646b9a84e9ca9298c874f669b55'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:42.085703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72738646b9a84e9ca9298c874f669b55'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:42.086587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72738646b9a84e9ca9298c874f669b55'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:42.087292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-957cfb96-cfb0-4c0c-a20b-60a1afabe6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72738646b9a84e9ca9298c874f669b55'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:42.100662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 888/1748] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:41 2022] DELETE /identity/v3/users/72738646b9a84e9ca9298c874f669b55 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:42.105262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177389-743a-4e34-881e-7186485c0007 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:42.105547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177389-743a-4e34-881e-7186485c0007 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:42.105819 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177389-743a-4e34-881e-7186485c0007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:42.127225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3b177389-743a-4e34-881e-7186485c0007 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:42.128585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3b177389-743a-4e34-881e-7186485c0007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:42.137105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 861/1749] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:42.147001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:42.148401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:42.206632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 889/1750] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:42.211772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b34253-be3e-4318-bc38-abfefc0b69de None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:42.216642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177389-743a-4e34-881e-7186485c0007 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:42.218083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3b177389-743a-4e34-881e-7186485c0007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:42.233598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:42.276160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:42.276787 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:42.277120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:42.277450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:42.288313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177389-743a-4e34-881e-7186485c0007 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:42.291992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:42.292605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:42.292887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:42.293196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:42.307311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:42.308799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:42.320055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 890/1751] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:42.350374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:42.351819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:42.362740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 862/1752] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:42.857550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:42.860234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:42.886389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280f50-3b97-407d-8737-2450a444816c service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:42.893058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:42.893714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:42.893988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:42.894254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] PATH_INFO: `/v3/projects/53dec18a0324410ba87e9565719299e3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:42.895413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=53dec18a0324410ba87e9565719299e3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:42.897402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:42.899890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:43.025518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53dec18a0324410ba87e9565719299e3', 'request_id': 'req-01f222a3-9b39-4493-82af-8cba12fa67bd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.035383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53dec18a0324410ba87e9565719299e3', 'request_id': 'req-01f222a3-9b39-4493-82af-8cba12fa67bd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.087564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53dec18a0324410ba87e9565719299e3', 'request_id': 'req-01f222a3-9b39-4493-82af-8cba12fa67bd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.102255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] The token cache is being invalidate because project 53dec18a0324410ba87e9565719299e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:43.102546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.103478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.104954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 891/1753] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:42 2022] DELETE /identity/v3/projects/53dec18a0324410ba87e9565719299e3 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:43.109732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:43.132641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce918a-c50b-4f24-8e24-f846cc8a406c service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:43.246248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:43.253217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:43.253217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:43.253217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] PATH_INFO: `/v3/users/329a994ae0ab4632a4dc6eb84af70c0e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:43.253217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=329a994ae0ab4632a4dc6eb84af70c0e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:43.253217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:43.435170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329a994ae0ab4632a4dc6eb84af70c0e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.447769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329a994ae0ab4632a4dc6eb84af70c0e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.527288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329a994ae0ab4632a4dc6eb84af70c0e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.527898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329a994ae0ab4632a4dc6eb84af70c0e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.528870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2dfa2c52-8d9a-4435-8d95-f9d41aac4525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329a994ae0ab4632a4dc6eb84af70c0e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:43.536124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:43.548643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 863/1754] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:43 2022] DELETE /identity/v3/users/329a994ae0ab4632a4dc6eb84af70c0e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:43.966184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:43.969630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:44.000719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f222a3-9b39-4493-82af-8cba12fa67bd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:44.063542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:44.064184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.064459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.064728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] PATH_INFO: `/v3/projects/0a85905e33c24df38fe52fe77916153c` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.065879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a85905e33c24df38fe52fe77916153c)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:44.067788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:44.174701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a85905e33c24df38fe52fe77916153c', 'request_id': 'req-96e2ce40-6148-45ea-8ff2-b050540bb456'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.182569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a85905e33c24df38fe52fe77916153c', 'request_id': 'req-96e2ce40-6148-45ea-8ff2-b050540bb456'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.259202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a85905e33c24df38fe52fe77916153c', 'request_id': 'req-96e2ce40-6148-45ea-8ff2-b050540bb456'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.278004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] The token cache is being invalidate because project 0a85905e33c24df38fe52fe77916153c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:44.278524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.279829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.285385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 892/1755] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:43 2022] DELETE /identity/v3/projects/0a85905e33c24df38fe52fe77916153c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:44.287704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0c32b-06de-491b-b7a3-8e2749c670a4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.288193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0c32b-06de-491b-b7a3-8e2749c670a4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.288288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0c32b-06de-491b-b7a3-8e2749c670a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.310008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-86a0c32b-06de-491b-b7a3-8e2749c670a4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:44.311219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-86a0c32b-06de-491b-b7a3-8e2749c670a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:44.391094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:44.392526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:44.437172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 864/1756] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:44.445205 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.445576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.449773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.458481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:44.507718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:44.512849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:44.533136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 865/1757] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:44.537425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:44.556059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7ef54-c81c-4c33-ae93-425e8ba834f9 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:44.565515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2ce40-6148-45ea-8ff2-b050540bb456 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:44.571386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:44.572090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.573333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.573854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:44.576643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] PATH_INFO: `/v3/users/4a9bdc4925c44fc7946fc3877c85993c` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.576766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.576766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.576932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a9bdc4925c44fc7946fc3877c85993c)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:44.578362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:44.578717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.579471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:44.579790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:44.581471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:44.609684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 866/1758] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:44.616863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:44.628220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08a80d-c328-4fd0-847b-51c6ad393f3a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:44.633327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:44.633943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.634285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.634565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.635323 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:44.637214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:44.743675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 867/1759] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:44 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:44.750524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:44.816433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a9bdc4925c44fc7946fc3877c85993c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.832834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5921e-b858-4e3c-a7b2-11da6825547b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:44.837913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:44.838665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.839091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.840004 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.841235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:44.843536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:44.876132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a9bdc4925c44fc7946fc3877c85993c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.894388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a9bdc4925c44fc7946fc3877c85993c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.895710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a9bdc4925c44fc7946fc3877c85993c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.896489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a32cb262-8db9-4721-896f-4e83c604baf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a9bdc4925c44fc7946fc3877c85993c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:44.922369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 893/1760] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:44 2022] DELETE /identity/v3/users/4a9bdc4925c44fc7946fc3877c85993c => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:44.926851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:44.928394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:44.928394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:44.951445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:44.952696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:44.955034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 868/1761] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:44 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:44.960233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:44.962389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:44.984351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f02dea-4966-4e81-bdf1-9bc8d04f8f5a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.048579 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 894/1762] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:45.057858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:45.061835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:45.063687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:45.083987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.085211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.085648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.091993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.091993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.091993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:45.109601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 869/1763] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:45.112174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:45.126848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bc161-09ad-413d-8daa-f0cd74ac92be admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.132933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.133601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.133882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.134256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.134918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.143570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:45.170140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 870/1764] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:45.171564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:45.182513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:45.190574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7cfc2-2a74-4f72-b1d0-a611ccd4f57d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.198626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e60fc-4653-46f5-9aa9-11e756641ca0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.202271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.206953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ce1ee-80e9-412a-88e9-8c64937d2c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.206953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ce1ee-80e9-412a-88e9-8c64937d2c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.206953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ce1ee-80e9-412a-88e9-8c64937d2c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.206953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ce1ee-80e9-412a-88e9-8c64937d2c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.207286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.207631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.208096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] PATH_INFO: `/v3/projects/1a5cb9d67e6349b68e116a897d2312d1/users/2f6096f562d14319a88d6562e520468f/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.226511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ce1ee-80e9-412a-88e9-8c64937d2c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.228833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ce1ee-80e9-412a-88e9-8c64937d2c9a service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:45.238026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5cb9d67e6349b68e116a897d2312d1, user_id=2f6096f562d14319a88d6562e520468f, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.243484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:45.248801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 895/1765] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:45.323620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 871/1766] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:45 2022] PUT /identity/v3/projects/1a5cb9d67e6349b68e116a897d2312d1/users/2f6096f562d14319a88d6562e520468f/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:45.338375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.338664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.338929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.376287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] MFA Rules not processed for user `2f6096f562d14319a88d6562e520468f`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:45.378903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:45.505988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 896/1767] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:45.520326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:45.522355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:45.623458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae76ba-5c6f-4baf-a4db-d19a135391cd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.633768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.639115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.639535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.639943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.654618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.656106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:45.673656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 872/1768] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:45.693589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:45.793326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:45.795852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:45.841386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474acb0-3297-4144-b645-a5bc6e052feb None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.857294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.857294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.857294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.857294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] PATH_INFO: `/v3/users/2f6096f562d14319a88d6562e520468f` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.857786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6096f562d14319a88d6562e520468f)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.857786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:45.975363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee54359-8a69-4015-86e8-f9a6d48b7ecb service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:45.983282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:45.987605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:45.987605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:45.987605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] PATH_INFO: `/v3/users/438997bb70d94306a15b08f6841f1bc4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:45.987605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=438997bb70d94306a15b08f6841f1bc4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:45.988729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:46.112523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6096f562d14319a88d6562e520468f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.171516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6096f562d14319a88d6562e520468f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.198048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438997bb70d94306a15b08f6841f1bc4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.203303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:46.208363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6096f562d14319a88d6562e520468f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.209199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6096f562d14319a88d6562e520468f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.209823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6096f562d14319a88d6562e520468f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.215681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:46.216838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438997bb70d94306a15b08f6841f1bc4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.230321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 897/1769] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:45 2022] DELETE /identity/v3/users/2f6096f562d14319a88d6562e520468f => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:46.231934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:46.235543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.281909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438997bb70d94306a15b08f6841f1bc4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.282848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438997bb70d94306a15b08f6841f1bc4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.283498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4e45b1c3-c3a3-402d-886b-a29530917cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438997bb70d94306a15b08f6841f1bc4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.297993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 873/1770] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:45 2022] DELETE /identity/v3/users/438997bb70d94306a15b08f6841f1bc4 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:46.301207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec323f-9e0a-46db-af2a-1e193e2f1715 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:46.301207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec323f-9e0a-46db-af2a-1e193e2f1715 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:46.301207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec323f-9e0a-46db-af2a-1e193e2f1715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:46.307227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49a2c-ab3d-4eb9-bbee-5609c2d012eb admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:46.321165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6aec323f-9e0a-46db-af2a-1e193e2f1715 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:46.322450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6aec323f-9e0a-46db-af2a-1e193e2f1715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.378935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:46.378935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:46.379234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:46.380595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] PATH_INFO: `/v3/projects/1911094c2ace4c29ace6678190932d46` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:46.381501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1911094c2ace4c29ace6678190932d46)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:46.383515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:46.427147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 874/1771] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:46.432959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:46.433447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:46.433967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:46.463917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:46.465164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.473968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 875/1772] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:46.474919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:46.476638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.553934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1911094c2ace4c29ace6678190932d46', 'request_id': 'req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.560388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c7eae-d03b-4eaa-aad1-ffdad3f9c3a2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:46.571646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1911094c2ace4c29ace6678190932d46', 'request_id': 'req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.633246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:46.633967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:46.634333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:46.634333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:46.717010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1911094c2ace4c29ace6678190932d46', 'request_id': 'req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.720001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:46.723662 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:46.725749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:46.733206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] The token cache is being invalidate because project 1911094c2ace4c29ace6678190932d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:46.733206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.735867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:46.738167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 876/1773] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:46.739261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:46.739410 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 898/1774] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:46 2022] DELETE /identity/v3/projects/1911094c2ace4c29ace6678190932d46 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:46.743782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.744823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:46.746792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.758688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53ff27-bed5-4f3a-8be8-99585c1bec05 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:46.764415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:46.764934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b52d4c-5cd4-4bb6-80f8-b4f918d5def9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:46.765056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:46.765357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:46.765639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:46.767791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:46.770518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:46.775173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:46.775173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:46.775173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] PATH_INFO: `/v3/users/80728cb0f4de42f0b5188698784cbafb` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:46.775173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80728cb0f4de42f0b5188698784cbafb)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:46.775173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:46.781979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:46.783256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:46.794713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 877/1775] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:46.912086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:47.030691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80728cb0f4de42f0b5188698784cbafb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.127826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80728cb0f4de42f0b5188698784cbafb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.144824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80728cb0f4de42f0b5188698784cbafb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.145556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80728cb0f4de42f0b5188698784cbafb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.146252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80728cb0f4de42f0b5188698784cbafb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.160130 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 899/1776] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:46 2022] DELETE /identity/v3/users/80728cb0f4de42f0b5188698784cbafb => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:47.410117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:47.411901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:47.450898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb4664-61fc-402b-bf4a-7ddda5417808 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:47.562315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:47.563122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:47.565224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:47.565940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] PATH_INFO: `/v3/projects/4300bdb9630f4101a602fd6e36982fb1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:47.569823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4300bdb9630f4101a602fd6e36982fb1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:47.572166 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:47.708702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:47.711590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:47.720219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4300bdb9630f4101a602fd6e36982fb1', 'request_id': 'req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.735210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c170c-c0b8-4cef-91d0-421d8e39c2d4 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:47.744448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4300bdb9630f4101a602fd6e36982fb1', 'request_id': 'req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.754693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4300bdb9630f4101a602fd6e36982fb1', 'request_id': 'req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.814016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:47.814968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:47.815314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:47.815651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] PATH_INFO: `/v3/projects/1a5cb9d67e6349b68e116a897d2312d1` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:47.816862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5cb9d67e6349b68e116a897d2312d1)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:47.818820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:47.837100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] The token cache is being invalidate because project 4300bdb9630f4101a602fd6e36982fb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:47.837632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.839190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:47.843320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 878/1777] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:47 2022] DELETE /identity/v3/projects/4300bdb9630f4101a602fd6e36982fb1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:48.005879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5cb9d67e6349b68e116a897d2312d1', 'request_id': 'req-96e7c691-a186-4934-92e5-dbab3841e834'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.014632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5cb9d67e6349b68e116a897d2312d1', 'request_id': 'req-96e7c691-a186-4934-92e5-dbab3841e834'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.098978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5cb9d67e6349b68e116a897d2312d1', 'request_id': 'req-96e7c691-a186-4934-92e5-dbab3841e834'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.106718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.108413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.112017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] The token cache is being invalidate because project 1a5cb9d67e6349b68e116a897d2312d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:48.112392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.113511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-96e7c691-a186-4934-92e5-dbab3841e834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.115175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 900/1778] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:47 2022] DELETE /identity/v3/projects/1a5cb9d67e6349b68e116a897d2312d1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:48.120070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dab5c-2614-4922-8d4e-c1741cba5744 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.120560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dab5c-2614-4922-8d4e-c1741cba5744 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.120834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dab5c-2614-4922-8d4e-c1741cba5744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.125296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f097-e0ea-4a97-9b8a-2b5dc0314ee1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.150437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-183dab5c-2614-4922-8d4e-c1741cba5744 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:48.151614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-183dab5c-2614-4922-8d4e-c1741cba5744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.182355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:48.201128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.201907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.202210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.202457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] PATH_INFO: `/v3/projects/11141ef34ba34be1ade403778fefdbe5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.203589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11141ef34ba34be1ade403778fefdbe5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.205924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.216879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 901/1779] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:48.224528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.224815 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.225128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.232581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:48.248044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:48.249413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.258345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 902/1780] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:48.262405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.273206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40142a1-17ef-48cd-a9e4-e8c89ef776e0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.285170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.286746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.287025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.294741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.294741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:48.294741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.294741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.305990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 903/1781] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:40:48.307351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.320356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0dd99-2b31-4d95-b4ea-002cbefffb0a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.328002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11141ef34ba34be1ade403778fefdbe5', 'request_id': 'req-809b600a-41d1-4987-881e-8a1fb2a886d4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.335940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11141ef34ba34be1ade403778fefdbe5', 'request_id': 'req-809b600a-41d1-4987-881e-8a1fb2a886d4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.342219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11141ef34ba34be1ade403778fefdbe5', 'request_id': 'req-809b600a-41d1-4987-881e-8a1fb2a886d4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.378932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375f7be-e2c0-4320-b81f-849803ced49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.379465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375f7be-e2c0-4320-b81f-849803ced49b admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.379747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375f7be-e2c0-4320-b81f-849803ced49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.380021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375f7be-e2c0-4320-b81f-849803ced49b admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.380642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375f7be-e2c0-4320-b81f-849803ced49b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.382687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375f7be-e2c0-4320-b81f-849803ced49b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.441161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] The token cache is being invalidate because project 11141ef34ba34be1ade403778fefdbe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:48.441450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.442069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-809b600a-41d1-4987-881e-8a1fb2a886d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.443669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 879/1782] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:48 2022] DELETE /identity/v3/projects/11141ef34ba34be1ade403778fefdbe5 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:48.445188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fcfe5-b460-4273-be78-863db04572a8 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.445188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fcfe5-b460-4273-be78-863db04572a8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.445461 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fcfe5-b460-4273-be78-863db04572a8 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.446816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 880/1783] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:40:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:40:48.450876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ea91-cec8-48c2-9703-f9da5264307c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.451089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ea91-cec8-48c2-9703-f9da5264307c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.451654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ea91-cec8-48c2-9703-f9da5264307c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.472931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1934ea91-cec8-48c2-9703-f9da5264307c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:48.474208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1934ea91-cec8-48c2-9703-f9da5264307c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.485271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 881/1784] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:48.493696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.493696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.493696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.509231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:48.510454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.517462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 904/1785] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/projects => generated 361 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:48.517626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.518041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.518471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.541709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:48.543429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.592104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 882/1786] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:48.593139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.595395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.619376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f05a8d-c27d-4f25-b98e-746c6d870d9f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.623225 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 905/1787] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:48.624137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.625655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.627851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.628468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.628743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.629074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.629676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.631595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.638884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b487834-cf24-4baa-91d7-d006ef7ab09a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.643774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.644492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.644788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.645095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] PATH_INFO: `/v3/users/a8706e96002946b2b9bfe87de5c3271a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.646230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8706e96002946b2b9bfe87de5c3271a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.648144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.724238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 883/1788] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:48 2022] POST /identity/v3/users => generated 281 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:48.724238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.724238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.793454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d27c-1bd6-4121-90dc-e6859667313d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.798538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.799144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.799445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.799713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.807815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.809981 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.819975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 884/1789] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:48.820732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.822178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:48.847416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4542d93-763d-431d-a81b-387962c8f3f9 service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:48.854110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8706e96002946b2b9bfe87de5c3271a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.855546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:48.856248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:48.856529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:48.856802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] PATH_INFO: `/v3/users/8fd9ccfc40744fa486d7a34d3b7dabbf` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:48.858038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd9ccfc40744fa486d7a34d3b7dabbf)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:48.859959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:48.939072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8706e96002946b2b9bfe87de5c3271a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.952724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8706e96002946b2b9bfe87de5c3271a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.956596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8706e96002946b2b9bfe87de5c3271a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.957280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8706e96002946b2b9bfe87de5c3271a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:48.969877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 906/1790] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:48 2022] DELETE /identity/v3/users/a8706e96002946b2b9bfe87de5c3271a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:48.970695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:48.972014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.054618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd9ccfc40744fa486d7a34d3b7dabbf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.068666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449c6f-add0-435d-bc70-7570ea4140d5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.069874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd9ccfc40744fa486d7a34d3b7dabbf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.076517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.077176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.078013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.078444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.079111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:49.079822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.081472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.095392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 907/1791] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:49.096538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.099658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.114212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327b8a7-1e8a-4af8-a745-2d6ff05d0526 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.119089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.119893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.120175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.121108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.121781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.124673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.149748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 908/1792] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:49.150716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.162268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe4ca6-31ee-4df4-a878-5daf585cdfe2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.167347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd9ccfc40744fa486d7a34d3b7dabbf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.167479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.167954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.168237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.168514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.168846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd9ccfc40744fa486d7a34d3b7dabbf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.169570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd9ccfc40744fa486d7a34d3b7dabbf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.169905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.172457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.184513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 885/1793] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:48 2022] DELETE /identity/v3/users/8fd9ccfc40744fa486d7a34d3b7dabbf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:49.188470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.190070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.253566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02bf5-658a-4259-be4b-a0cb44609b4c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.258424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb400b-c8e2-48c1-bf9a-125ad3f654e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.259033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb400b-c8e2-48c1-bf9a-125ad3f654e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.259310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb400b-c8e2-48c1-bf9a-125ad3f654e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.259581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb400b-c8e2-48c1-bf9a-125ad3f654e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.260215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb400b-c8e2-48c1-bf9a-125ad3f654e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.261858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb400b-c8e2-48c1-bf9a-125ad3f654e3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.272653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 886/1794] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:49.273874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.279711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.279711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.293225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:49.294455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.302240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 887/1795] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:49.303276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.317701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 909/1796] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:49 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:49.320427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.324716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a319f6-f071-4c28-8cc8-c93608a1f9cb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.329715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.386111 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.386782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.387059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.387330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] PATH_INFO: `/v3/projects/504374ae50f34599b0eae6f122e520c1/users/914a10f53fc44307824bc81e7fc7f9dd/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.397851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f198-8e2f-4776-8839-e4b7f0e88a98 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.401903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.402036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] RBAC: Authorizing `identity:create_grant(project_id=504374ae50f34599b0eae6f122e520c1, user_id=914a10f53fc44307824bc81e7fc7f9dd, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.402509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.402790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.403607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.405485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.413675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.415042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.424848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 910/1797] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:49.425643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.426925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.440193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f97d-0b70-4753-8f19-6520d1ec97a1 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.444840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60145db3-d979-4bf4-90b9-2dc9fae19cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.445472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145db3-d979-4bf4-90b9-2dc9fae19cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.445759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145db3-d979-4bf4-90b9-2dc9fae19cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.446021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145db3-d979-4bf4-90b9-2dc9fae19cbf admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.446663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60145db3-d979-4bf4-90b9-2dc9fae19cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.448614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60145db3-d979-4bf4-90b9-2dc9fae19cbf admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.461097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 888/1798] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:49 2022] PUT /identity/v3/projects/504374ae50f34599b0eae6f122e520c1/users/914a10f53fc44307824bc81e7fc7f9dd/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:49.461994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.463693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.537745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a56956-d372-4945-b614-95ce46bb457c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.542478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.543109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.543383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.544291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] PATH_INFO: `/v3/projects/b60403061fba42eba98a7c89987672a8` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.545202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60403061fba42eba98a7c89987672a8)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.547210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.550891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 911/1799] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:49 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:49.552206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.552647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.553828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.574481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:49.585896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] MFA Rules not processed for user `914a10f53fc44307824bc81e7fc7f9dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:49.587140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.654830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 912/1800] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:49.655618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.667981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27031d2-b13b-4570-8610-cbcd0b377091 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.690590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60403061fba42eba98a7c89987672a8', 'request_id': 'req-a0b34aaa-2d33-4678-aef3-ade53965b2a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.698702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60403061fba42eba98a7c89987672a8', 'request_id': 'req-a0b34aaa-2d33-4678-aef3-ade53965b2a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.720486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60403061fba42eba98a7c89987672a8', 'request_id': 'req-a0b34aaa-2d33-4678-aef3-ade53965b2a1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.740938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.741598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.741878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.742155 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.742802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.744529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.748779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:49.757213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 913/1801] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:49.758184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.759846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.813243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] The token cache is being invalidate because project b60403061fba42eba98a7c89987672a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:49.813531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.814303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:49.819490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 889/1802] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:49 2022] DELETE /identity/v3/projects/b60403061fba42eba98a7c89987672a8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:49.819857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4756583-d1c8-4fbb-8efb-a82bb662423f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.820431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.821987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.824192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.824821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.825761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.826370 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.827739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.836135 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b34aaa-2d33-4678-aef3-ade53965b2a1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.841508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e8def-5c8c-4012-90b9-4ffe00e294d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.842030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8def-5c8c-4012-90b9-4ffe00e294d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.842306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8def-5c8c-4012-90b9-4ffe00e294d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.842598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8def-5c8c-4012-90b9-4ffe00e294d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.843223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e8def-5c8c-4012-90b9-4ffe00e294d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.844830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e8def-5c8c-4012-90b9-4ffe00e294d4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.855217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 890/1803] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:49.859262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.859262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.859262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.878613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:49.879904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.894327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 891/1804] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:49.894327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.897696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.898421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108dfa-d689-4873-8966-7ad810d1c219 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.899161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03bf0c-fac9-4c69-a1da-e04c0368dd92 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.903309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.904043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.904245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.904517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] PATH_INFO: `/v3/projects/eabd08343b0f49049685e92f9a45a796/users/0b24b14bd8484203a1a1861e8987573b/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.909805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 914/1805] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:49.911997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.911997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.911997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.923792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eabd08343b0f49049685e92f9a45a796, user_id=0b24b14bd8484203a1a1861e8987573b, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.926595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:49.938805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:49.940002 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.949611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 915/1806] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:49.951199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:49.952483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:49.966533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af47cc3-4087-4246-925b-e594dc0f8183 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:49.971488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:49.972149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:49.972436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:49.972791 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] PATH_INFO: `/v3/users/914a10f53fc44307824bc81e7fc7f9dd` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:49.981666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914a10f53fc44307824bc81e7fc7f9dd)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:49.981666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.030764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 892/1807] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:49 2022] PUT /identity/v3/projects/eabd08343b0f49049685e92f9a45a796/users/0b24b14bd8484203a1a1861e8987573b/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:50.031743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.033522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.099415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357547b-fccb-45b7-964f-4bea825334e2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.108101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.108729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.108993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.109294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] PATH_INFO: `/v3/projects/0c51788c68ee45b99646d4699d17a28c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.110619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c51788c68ee45b99646d4699d17a28c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.112542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.165219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914a10f53fc44307824bc81e7fc7f9dd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.251272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c51788c68ee45b99646d4699d17a28c', 'request_id': 'req-ac859866-3695-46fe-a6a0-c74edc902da6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.258191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c51788c68ee45b99646d4699d17a28c', 'request_id': 'req-ac859866-3695-46fe-a6a0-c74edc902da6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.265444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c51788c68ee45b99646d4699d17a28c', 'request_id': 'req-ac859866-3695-46fe-a6a0-c74edc902da6'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.281659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914a10f53fc44307824bc81e7fc7f9dd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.295494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914a10f53fc44307824bc81e7fc7f9dd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.296278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914a10f53fc44307824bc81e7fc7f9dd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.296924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914a10f53fc44307824bc81e7fc7f9dd'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.310530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 916/1808] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:49 2022] DELETE /identity/v3/users/914a10f53fc44307824bc81e7fc7f9dd => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:50.313810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.313810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.377515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627b1f-963b-4075-aba6-e9472e78b286 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.382177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.382784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.383098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.383371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.384010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:50.384312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.385940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d83816-9ff1-47bb-ad78-25d4f9981ea0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.397497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 917/1809] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:50.398652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3f100-984a-4686-b02a-61726c272275 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.398979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3f100-984a-4686-b02a-61726c272275 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.399252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3f100-984a-4686-b02a-61726c272275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.405151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] The token cache is being invalidate because project 0c51788c68ee45b99646d4699d17a28c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:50.405449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.406614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ac859866-3695-46fe-a6a0-c74edc902da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:50.408550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 893/1810] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:50 2022] DELETE /identity/v3/projects/0c51788c68ee45b99646d4699d17a28c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:50.410925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.410925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.410925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.429836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:50.431034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.433162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-85c3f100-984a-4686-b02a-61726c272275 None None] MFA Rules not processed for user `0b24b14bd8484203a1a1861e8987573b`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:50.434658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-85c3f100-984a-4686-b02a-61726c272275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.440296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 894/1811] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:50.441128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.444337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.456428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403d997-e31b-43bb-ad29-b241db14bcc3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.460962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.461594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.462160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.462160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.462742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.464643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.504776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 918/1812] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:50.505965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.506246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.506518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.527101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:50.528992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.536699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 919/1813] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:50.537569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.538657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.580902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 895/1814] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:50.582067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.585656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.596611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739814-0604-40e4-9eb6-9cabaf0bafd4 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.647612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebad0bd-4644-457e-a76d-14504b067e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.648223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebad0bd-4644-457e-a76d-14504b067e23 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.648505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebad0bd-4644-457e-a76d-14504b067e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.648770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebad0bd-4644-457e-a76d-14504b067e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.651043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f84210-6b5f-4839-9d31-c77d69239608 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.655840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.656450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.656729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.657018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.657696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.658518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebad0bd-4644-457e-a76d-14504b067e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.660055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.660209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebad0bd-4644-457e-a76d-14504b067e23 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.670622 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 920/1815] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:50.672003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.672309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.672568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.693700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:50.694924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:50.703985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 921/1816] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:50.705028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.708718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:50.714779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6ac11-3499-4dd2-a7e5-6472dd166949 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.719788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.720564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.720839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.721146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.721776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.723680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.763638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 896/1817] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:50.764458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.824847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d88d-c1e0-454c-ba21-1e80fd45cf8c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.830671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.831302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.831613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.831887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.832538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:50.832841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.834575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.839065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 922/1818] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:50.839065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.846632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01c007-227e-46fc-a556-e5c9271c9420 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.852789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 897/1819] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:50.852789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.853966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.857565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.857565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.857565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.857565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.858004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.863224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed000b7-6a0d-4707-a99c-1eef42a339b7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.867910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.868505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.868779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.869080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.869712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.871272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.876885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:50.881975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 898/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:50.883971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.896842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc6ca7-3bb0-4c64-b77e-6412d502d9d8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.901687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.902278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.902555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.902816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.903784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.905690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.945089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 923/1821] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:50.946121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.956341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cadc53-8083-4ebc-ba7e-4b12d3bfa5ec admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.961291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.961888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.962222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.962492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.963126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.964702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:50.976786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 924/1822] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:50.977774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:50.989941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542364f9-512a-4b4e-9459-80bc244a16f0 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:50.994556 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:50.995151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:50.995426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:50.995698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:50.996330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:50.998493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.008498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 925/1823] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:51.009382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.024084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55013e33-884d-4e5b-afce-da5dd59abd8e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.024476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 899/1824] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:50 2022] POST /identity/v3/projects => generated 397 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:51.024476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.081820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.082582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.082876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.083172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] PATH_INFO: `/v3/projects/d2eb08b7b586486e8ff815c65419c1f4/users/928f4340b3e646f9b1a232160609fa7b/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.091780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899fac8-33a8-43d2-aca3-df9bc7e8f9c1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.096524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.097202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.098875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.098875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] PATH_INFO: `/v3/projects/f5e60c9ad6b94dea906de2f2641eb0a4/users/2e594ca404814cce936b6815d6a796f1/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.099062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2eb08b7b586486e8ff815c65419c1f4, user_id=928f4340b3e646f9b1a232160609fa7b, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.100797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.112455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e60c9ad6b94dea906de2f2641eb0a4, user_id=2e594ca404814cce936b6815d6a796f1, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.115132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.209078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 926/1825] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:51 2022] PUT /identity/v3/projects/d2eb08b7b586486e8ff815c65419c1f4/users/928f4340b3e646f9b1a232160609fa7b/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:51.218532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.219314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 900/1826] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:51 2022] PUT /identity/v3/projects/f5e60c9ad6b94dea906de2f2641eb0a4/users/2e594ca404814cce936b6815d6a796f1/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:51.220781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.224127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.284875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8423b17-8ae1-4ab9-a646-4016c0f1c260 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.289773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde27e-ca72-4ccd-935c-f8f60740c583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.290511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde27e-ca72-4ccd-935c-f8f60740c583 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.290773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde27e-ca72-4ccd-935c-f8f60740c583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.291043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde27e-ca72-4ccd-935c-f8f60740c583 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.291175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231f3f-d03c-4edf-97c5-087b8530da32 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.291680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddde27e-ca72-4ccd-935c-f8f60740c583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.294256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddde27e-ca72-4ccd-935c-f8f60740c583 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.296900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.297681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.297973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.298250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] PATH_INFO: `/v3/projects/504374ae50f34599b0eae6f122e520c1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.299552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=504374ae50f34599b0eae6f122e520c1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.301531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.380077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 927/1827] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:51 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:51.381586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.381938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.382280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.413576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504374ae50f34599b0eae6f122e520c1', 'request_id': 'req-4c08ca30-4e17-4a94-a514-fdd80fe439ef'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:51.413921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] MFA Rules not processed for user `928f4340b3e646f9b1a232160609fa7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:51.415136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.422153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504374ae50f34599b0eae6f122e520c1', 'request_id': 'req-4c08ca30-4e17-4a94-a514-fdd80fe439ef'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:51.428209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504374ae50f34599b0eae6f122e520c1', 'request_id': 'req-4c08ca30-4e17-4a94-a514-fdd80fe439ef'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:51.484171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 928/1828] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:51.486412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.533350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] The token cache is being invalidate because project 504374ae50f34599b0eae6f122e520c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:51.533925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:51.534414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:51.536067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 901/1829] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:51 2022] DELETE /identity/v3/projects/504374ae50f34599b0eae6f122e520c1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:51.540520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.540520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.551094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806484f6-75c6-4b27-a300-fcfae368595c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.555795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.556420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.556704 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.557061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.557740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.559875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.570160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 929/1830] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:51.570826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.572522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.585472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:51.611401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08ca30-4e17-4a94-a514-fdd80fe439ef admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.616410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c6a95-33cd-4460-a8d2-70d67ef40dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.617047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c6a95-33cd-4460-a8d2-70d67ef40dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.617332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c6a95-33cd-4460-a8d2-70d67ef40dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.617606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c6a95-33cd-4460-a8d2-70d67ef40dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.618382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c6a95-33cd-4460-a8d2-70d67ef40dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.620066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c6a95-33cd-4460-a8d2-70d67ef40dc7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.634985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 902/1831] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:51.634985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.634985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.634985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.649536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160b56-643f-4380-b6e7-439fcb0da810 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.654538 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:51.654912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:51.656162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.657604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.658214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.658492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.666100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.666100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.673636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 903/1832] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:51.673636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.673636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.673937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.674051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.680726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41d21a-850a-4f1c-a5d2-654fe131ed1d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.683477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 930/1833] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:51.684802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.686762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0cb82-8a84-4a3c-8282-0f04a7e3690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.689108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0cb82-8a84-4a3c-8282-0f04a7e3690a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.689108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0cb82-8a84-4a3c-8282-0f04a7e3690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.689108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0cb82-8a84-4a3c-8282-0f04a7e3690a admin admin] PATH_INFO: `/v3/projects/f5e60c9ad6b94dea906de2f2641eb0a4/users/2e594ca404814cce936b6815d6a796f1/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.690595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0cb82-8a84-4a3c-8282-0f04a7e3690a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e60c9ad6b94dea906de2f2641eb0a4, user_id=2e594ca404814cce936b6815d6a796f1, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.692906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0cb82-8a84-4a3c-8282-0f04a7e3690a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.695281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe765f-ebae-4f0f-bfbf-12088025d8f4 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.700380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557acb9-fdfd-458e-869e-eb6294ad6d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.700980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557acb9-fdfd-458e-869e-eb6294ad6d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.701282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557acb9-fdfd-458e-869e-eb6294ad6d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.701562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557acb9-fdfd-458e-869e-eb6294ad6d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.702202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557acb9-fdfd-458e-869e-eb6294ad6d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.703795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557acb9-fdfd-458e-869e-eb6294ad6d26 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.706551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 904/1834] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:51 2022] PUT /identity/v3/projects/f5e60c9ad6b94dea906de2f2641eb0a4/users/2e594ca404814cce936b6815d6a796f1/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:51.707731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.708080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.708418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.725373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 931/1835] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:51.728640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.728640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.728640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.735245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:51.737414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.760987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] MFA Rules not processed for user `2e594ca404814cce936b6815d6a796f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:51.764213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:51.765238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.804839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 905/1836] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:51.805843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.816488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e328f96-1c0d-42e2-ad76-02854ccbca7b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.822549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.823820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.824105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.824372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] PATH_INFO: `/v3/projects/dd46529250d64c05a4f869388c40cc57/users/64092af98e61463784ca2c7f359d8b90/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.841857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd46529250d64c05a4f869388c40cc57, user_id=64092af98e61463784ca2c7f359d8b90, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.844539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.852971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 932/1837] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:51.854205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.864176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c2954-77cb-4f65-a7e9-c0faebb4e5b9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.868694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.869316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.872928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.872928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.872928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:51.872928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.872928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:51.885078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 933/1838] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:51.885412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.887084 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:51.889532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:51.917528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 906/1839] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:51 2022] PUT /identity/v3/projects/dd46529250d64c05a4f869388c40cc57/users/64092af98e61463784ca2c7f359d8b90/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:51.928739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:51.950025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567bea4-f9ff-4007-b1b9-5231d3ab3d91 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.992537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de01db-9b79-418c-a9fe-dbc61fa2560a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:51.997431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:51.998222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:51.998494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:51.998759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:51.999407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:51.999516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f9f06-db5b-4d79-b019-a699a2ba96b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.000217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f9f06-db5b-4d79-b019-a699a2ba96b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.000504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f9f06-db5b-4d79-b019-a699a2ba96b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.000859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f9f06-db5b-4d79-b019-a699a2ba96b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.001418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.066301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f9f06-db5b-4d79-b019-a699a2ba96b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.067700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f9f06-db5b-4d79-b019-a699a2ba96b8 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.077929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 934/1840] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:52.079275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.079569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.079843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.093295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 907/1841] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:51 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:52.094859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:52.095466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:52.113414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] MFA Rules not processed for user `64092af98e61463784ca2c7f359d8b90`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:52.114731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:52.167730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379505a-a879-475c-8d00-0c6ed07bd029 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:52.173622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.174235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.174516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.174791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.176125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.178348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.182247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 935/1842] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:52.197821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:52.256015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63442a-ddbb-42ff-8071-4956ac7a7926 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:52.260747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.261384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.261662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.261939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.271335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.272770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.283717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 936/1843] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:52 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:40:52.293091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 908/1844] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:52 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:52.294808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:52.296753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:52.315143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30d4fd-8722-40fe-8b39-5fb58d91297f service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:52.320436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.321150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.321433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.321706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.322427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.324149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.336748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 937/1845] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:52.340737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:52.353060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efaacc-6b1a-4886-bad0-81bd2697433d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:52.359186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97544cc0-e1fa-4735-99a6-c5d99642e08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.359799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97544cc0-e1fa-4735-99a6-c5d99642e08f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.360080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97544cc0-e1fa-4735-99a6-c5d99642e08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.360351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97544cc0-e1fa-4735-99a6-c5d99642e08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.360993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97544cc0-e1fa-4735-99a6-c5d99642e08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.362638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97544cc0-e1fa-4735-99a6-c5d99642e08f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.386617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 909/1846] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:40:52.386995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:52.405880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029debe0-31ce-4b93-8dc4-886d6396b978 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:52.426907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:52.431636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.432752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.433047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.433749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] PATH_INFO: `/v3/projects/5528efbb3dcf4a3fbc892cb01540ca66/users/c854907702fe41a3b89b858bb6da0e8e/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.470026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5528efbb3dcf4a3fbc892cb01540ca66, user_id=c854907702fe41a3b89b858bb6da0e8e, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.474979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.558323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 938/1847] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:52 2022] PUT /identity/v3/projects/5528efbb3dcf4a3fbc892cb01540ca66/users/c854907702fe41a3b89b858bb6da0e8e/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:40:52.569806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.570285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.570285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.614739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] MFA Rules not processed for user `c854907702fe41a3b89b858bb6da0e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:52.615962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:52.692656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 910/1848] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:52.706381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:52.708192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:52.740347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:52.779133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b25ce-1c04-4d4c-adad-2e9218eda81a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:52.782894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5644ab2-2d32-4a4e-97a4-0ba3d7477b10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:52.783499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644ab2-2d32-4a4e-97a4-0ba3d7477b10 service glance] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:52.783774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644ab2-2d32-4a4e-97a4-0ba3d7477b10 service glance] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:52.784049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644ab2-2d32-4a4e-97a4-0ba3d7477b10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:52.792181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644ab2-2d32-4a4e-97a4-0ba3d7477b10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:52.793683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644ab2-2d32-4a4e-97a4-0ba3d7477b10 service glance] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:52.802470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 939/1849] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:52.968285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:53.139771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:53.384857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:55.813388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:55.880960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0d2e6-51a2-47ac-bf4c-6e1d2065f1e5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:55.886015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:55.889951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:55.889951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:55.889951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] PATH_INFO: `/v3/users/0b24b14bd8484203a1a1861e8987573b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:55.889951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b24b14bd8484203a1a1861e8987573b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:55.891680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:56.198183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b24b14bd8484203a1a1861e8987573b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:56.211279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b24b14bd8484203a1a1861e8987573b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:56.281565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b24b14bd8484203a1a1861e8987573b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:56.284650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b24b14bd8484203a1a1861e8987573b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:56.284650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b24b14bd8484203a1a1861e8987573b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:56.296897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 911/1850] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:55 2022] DELETE /identity/v3/users/0b24b14bd8484203a1a1861e8987573b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:56.301508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:56.301959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:56.302047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:56.321263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:56.322449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:56.382352 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 940/1851] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:56.392343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:56.397399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:56.437579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:56.454282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471882e-54af-402c-86f2-bc02b10d40d1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:56.457866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:56.458758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:56.458758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:56.458758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:56.468113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:56.469587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:56.479117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 912/1852] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:40:56.854745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:56.856205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:56.871263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686445-602f-4f56-b75c-233dea82e2d3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:56.877582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:56.878594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:56.878883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:56.879164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] PATH_INFO: `/v3/projects/eabd08343b0f49049685e92f9a45a796` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:56.880334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eabd08343b0f49049685e92f9a45a796)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:56.882558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:56.884219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:56.990594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabd08343b0f49049685e92f9a45a796', 'request_id': 'req-03f10156-7ca0-49a3-806c-977a42ea1a40'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:56.996931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabd08343b0f49049685e92f9a45a796', 'request_id': 'req-03f10156-7ca0-49a3-806c-977a42ea1a40'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.076028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabd08343b0f49049685e92f9a45a796', 'request_id': 'req-03f10156-7ca0-49a3-806c-977a42ea1a40'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.087377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] The token cache is being invalidate because project eabd08343b0f49049685e92f9a45a796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:57.087665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.088288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.089820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 941/1853] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:56 2022] DELETE /identity/v3/projects/eabd08343b0f49049685e92f9a45a796 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:57.094104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:57.104027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51153d4-33d4-4e8d-917b-60c4f573802a service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:57.168293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:57.168293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:57.168293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:57.168293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] PATH_INFO: `/v3/users/2e594ca404814cce936b6815d6a796f1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:57.168780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e594ca404814cce936b6815d6a796f1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:57.168780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:57.388006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e594ca404814cce936b6815d6a796f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.401251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e594ca404814cce936b6815d6a796f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.500367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e594ca404814cce936b6815d6a796f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.501075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e594ca404814cce936b6815d6a796f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.501774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-98dd757b-228c-4d49-a427-2940216bd722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e594ca404814cce936b6815d6a796f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:57.515344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 913/1854] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:40:57 2022] DELETE /identity/v3/users/2e594ca404814cce936b6815d6a796f1 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:57.986772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:57.988303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:58.002298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f10156-7ca0-49a3-806c-977a42ea1a40 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.065714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.066346 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.066625 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.066893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] PATH_INFO: `/v3/projects/f5e60c9ad6b94dea906de2f2641eb0a4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.067999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e60c9ad6b94dea906de2f2641eb0a4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.069935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:58.181149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e60c9ad6b94dea906de2f2641eb0a4', 'request_id': 'req-bea457ec-2e10-4d6d-b583-bc53e7035f8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:58.190692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e60c9ad6b94dea906de2f2641eb0a4', 'request_id': 'req-bea457ec-2e10-4d6d-b583-bc53e7035f8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:58.305056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e60c9ad6b94dea906de2f2641eb0a4', 'request_id': 'req-bea457ec-2e10-4d6d-b583-bc53e7035f8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:58.318150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] The token cache is being invalidate because project f5e60c9ad6b94dea906de2f2641eb0a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:40:58.320145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:58.320145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bea457ec-2e10-4d6d-b583-bc53e7035f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:40:58.320568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 942/1855] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:40:57 2022] DELETE /identity/v3/projects/f5e60c9ad6b94dea906de2f2641eb0a4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:58.325487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.326089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.326693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.347382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:58.348639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:58.419297 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:58.429725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 914/1856] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:58.442759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.442759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.442759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.461175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:58.462523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:58.471292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 943/1857] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:58.477459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:58.491003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274b4ae-1901-45d2-bebc-2493c398bb95 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.495808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.496444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.496722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.496994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.497733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:40:58.498075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.499664 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:58.511380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 915/1858] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:40:58.516337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:58.526292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349813a-cc29-4696-87ee-9b29d3a6d239 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.531807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.532393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.532669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.532936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.533592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.535478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:58.551690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:58.638126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 944/1859] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:58 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:58.642935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:58.655121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1fa20-883b-499a-85b0-6a5745ada783 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.717299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.718029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.718213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.718486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.719162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.721177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:58.843341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 916/1860] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:58 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:58.847304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:58.857636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08f871-13c6-4317-8ae6-186ebe5b325a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.862129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.862727 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.863001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.863268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.863909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.865516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:58.875877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 945/1861] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:58.879945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:58.890946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09b8a8-8e86-41a9-81eb-9ab14a4a905c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.895664 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88456097-65a1-441c-ae10-cc1583cda4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.896264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88456097-65a1-441c-ae10-cc1583cda4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.896536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88456097-65a1-441c-ae10-cc1583cda4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.896801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88456097-65a1-441c-ae10-cc1583cda4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.897608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88456097-65a1-441c-ae10-cc1583cda4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.899231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88456097-65a1-441c-ae10-cc1583cda4b0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:58.913619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 917/1862] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:58.917787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:58.927977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdd0e6-d147-4905-8a8c-0d2c63bab876 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:58.932464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:58.933150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:58.933426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:58.933701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] PATH_INFO: `/v3/projects/d95bb59c3bd34c30b86182774de4df2b/users/ac9c12fecc044219b431e8dc3a822bb8/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:58.948806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95bb59c3bd34c30b86182774de4df2b, user_id=ac9c12fecc044219b431e8dc3a822bb8, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:58.952147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.042334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 946/1863] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:58 2022] PUT /identity/v3/projects/d95bb59c3bd34c30b86182774de4df2b/users/ac9c12fecc044219b431e8dc3a822bb8/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:59.053496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.053781 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.054053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.085394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] MFA Rules not processed for user `ac9c12fecc044219b431e8dc3a822bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:59.086930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:59.158306 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 918/1864] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:40:59.162967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.230954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f1a65-4b71-41f0-81ae-8990ecfaf978 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.236557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.237276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.237553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.237823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.238461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.240439 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.339609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 947/1865] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:40:59 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:40:59.344278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.355351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171796e6-964d-4f81-adca-6bf0be79de7d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.402500 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:59.411839 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.412449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.412729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.412994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.413640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.415540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.488006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 919/1866] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:40:59 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:40:59.492023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.502510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97bc5-fede-4239-8927-ac5b9be9f7ee admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.507238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.507856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.508136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.508406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.509108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.510693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.522248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 948/1867] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:59.526557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.537621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e479-6881-4b45-b21d-e0840d04bc82 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.542339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.542982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.543257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.543667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] PATH_INFO: `/v3/projects/f20120e071fe41249aefbaf1e9c92e5d/users/36ad43b19dc04639ad38fe2e9e2d05f4/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.558379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20120e071fe41249aefbaf1e9c92e5d, user_id=36ad43b19dc04639ad38fe2e9e2d05f4, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.561706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.602128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 920/1868] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:59 2022] PUT /identity/v3/projects/f20120e071fe41249aefbaf1e9c92e5d/users/36ad43b19dc04639ad38fe2e9e2d05f4/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:59.606633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.618135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525adac-ce4f-41ac-bd46-25dbc4b7344b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.676025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28cdbe-a903-41d4-bec1-f1cb42024aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.676624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28cdbe-a903-41d4-bec1-f1cb42024aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.676899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28cdbe-a903-41d4-bec1-f1cb42024aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.677198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28cdbe-a903-41d4-bec1-f1cb42024aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.677827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28cdbe-a903-41d4-bec1-f1cb42024aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.679399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28cdbe-a903-41d4-bec1-f1cb42024aa6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.689157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 949/1869] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:40:59.693517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.704570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99132d5a-a031-4700-bc29-8810fd10f1f8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.709431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.710322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.710600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.710876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] PATH_INFO: `/v3/projects/f20120e071fe41249aefbaf1e9c92e5d/users/36ad43b19dc04639ad38fe2e9e2d05f4/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.712790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20120e071fe41249aefbaf1e9c92e5d, user_id=36ad43b19dc04639ad38fe2e9e2d05f4, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.715642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.726875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 921/1870] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:40:59 2022] PUT /identity/v3/projects/f20120e071fe41249aefbaf1e9c92e5d/users/36ad43b19dc04639ad38fe2e9e2d05f4/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:40:59.737336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.737674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.741319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.769889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] MFA Rules not processed for user `36ad43b19dc04639ad38fe2e9e2d05f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:40:59.771082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:59.784060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:40:59.840903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 950/1871] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:40:59.851886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:40:59.853663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:40:59.916875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f13f6-f2fa-4e1e-ae56-d6bfedcfeb1e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:40:59.920706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951511b-21a3-477e-8506-6ac35f1cd7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:40:59.921338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951511b-21a3-477e-8506-6ac35f1cd7ab service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:40:59.921614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951511b-21a3-477e-8506-6ac35f1cd7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:40:59.921888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951511b-21a3-477e-8506-6ac35f1cd7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:40:59.980508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951511b-21a3-477e-8506-6ac35f1cd7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:40:59.981966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951511b-21a3-477e-8506-6ac35f1cd7ab service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:40:59.990713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 922/1872] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:03.050890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:03.052450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:03.123477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a09b-491c-48d5-8513-ae73a0fdea57 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:03.128289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:03.128920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:03.129232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:03.129506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] PATH_INFO: `/v3/users/b94f3cd454634df486808360afa3de83` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:03.130621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94f3cd454634df486808360afa3de83)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:03.132652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:03.340645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94f3cd454634df486808360afa3de83'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:03.439115 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94f3cd454634df486808360afa3de83'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:03.450034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94f3cd454634df486808360afa3de83'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:03.450864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94f3cd454634df486808360afa3de83'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:03.451484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94f3cd454634df486808360afa3de83'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:03.464299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 951/1873] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:03 2022] DELETE /identity/v3/users/b94f3cd454634df486808360afa3de83 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:03.469385 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:03.469904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:03.470007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:03.492827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:03.494084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:03.549379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:03.558419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 923/1874] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:03.570364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:03.572468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:03.620854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:03.646982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74cfe3-f81b-4044-bdda-e13d438c26db admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:03.650558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:03.651205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:03.651476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:03.651771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:03.661883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:03.663488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:03.673659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 952/1875] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:04.124465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:04.126367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.150386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d95be2-b64e-4be9-b818-2719544e8c58 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:04.160835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:04.161889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.162354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.162597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] PATH_INFO: `/v3/projects/25f0bbc5c14b4fc496443813be821dfc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.163842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f0bbc5c14b4fc496443813be821dfc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:04.165918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:04.247953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:04.247953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.255523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7277a-92d3-47cc-a2e3-d04fc4e09b73 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:04.260499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:04.261161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.261440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.261711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] PATH_INFO: `/v3/users/928f4340b3e646f9b1a232160609fa7b` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.262857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928f4340b3e646f9b1a232160609fa7b)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:04.265287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:04.316884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f0bbc5c14b4fc496443813be821dfc', 'request_id': 'req-d1726e9e-e68a-42af-b03c-11ca9e806d4f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.323176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f0bbc5c14b4fc496443813be821dfc', 'request_id': 'req-d1726e9e-e68a-42af-b03c-11ca9e806d4f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.328761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f0bbc5c14b4fc496443813be821dfc', 'request_id': 'req-d1726e9e-e68a-42af-b03c-11ca9e806d4f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.419278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] The token cache is being invalidate because project 25f0bbc5c14b4fc496443813be821dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:04.419568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.420759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1726e9e-e68a-42af-b03c-11ca9e806d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.422296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 924/1876] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:04 2022] DELETE /identity/v3/projects/25f0bbc5c14b4fc496443813be821dfc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:04.426821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33c0d-2b5e-4181-9fa3-f2a163b58994 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.427126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33c0d-2b5e-4181-9fa3-f2a163b58994 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.427404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33c0d-2b5e-4181-9fa3-f2a163b58994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.447692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-83d33c0d-2b5e-4181-9fa3-f2a163b58994 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:04.451530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-83d33c0d-2b5e-4181-9fa3-f2a163b58994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.486491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928f4340b3e646f9b1a232160609fa7b'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.511573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 925/1877] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:04.519392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.519674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.519999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.540585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:04.541799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.550765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 926/1878] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:04.554886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:04.556794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928f4340b3e646f9b1a232160609fa7b'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.564160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b9f5-0974-415a-97bd-08f0fa378666 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:04.569687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:04.574518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.574518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.574747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928f4340b3e646f9b1a232160609fa7b'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.574843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.574935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928f4340b3e646f9b1a232160609fa7b'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.575026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:04.575026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:04.575197 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-642504d9-3004-4f56-9704-1a982de52e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928f4340b3e646f9b1a232160609fa7b'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:04.575288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:04.584933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 953/1879] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:04 2022] DELETE /identity/v3/users/928f4340b3e646f9b1a232160609fa7b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:04.586482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 927/1880] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:41:04.589471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.589673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.589821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.593130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:04.595064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.608902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42a72-99d2-416b-a00a-002def260997 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:04.615131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:04.616794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.674216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:04.674840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.675122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.675397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.676031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:04.678148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:04.692651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:04.705541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 954/1881] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:04.715328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:04.716839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:04.774474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:04.780347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb70e4b-c174-4bdc-8e46-34f80c3c5759 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:04.784057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:04.784692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.784974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.785277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.794441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:04.796340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:04.805148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 928/1882] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:04 2022] POST /identity/v3/projects => generated 371 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:04.805974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 955/1883] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:04.809512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:04.884836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861a4a4-3dcc-4542-a39d-26f18d2aa61c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:04.891249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:04.892170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:04.892602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:04.893034 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:04.893817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:04.896009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:04.996928 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 929/1884] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:04 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:05.002108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.011042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e769-1a95-46e9-a429-30040b9bbcb4 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.016394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.017153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.017570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.017851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.018486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.020069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.031438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 956/1885] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:05.035720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.050276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93096-732e-47ac-806e-c38877182c98 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.054980 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b567d-9072-48ec-a9be-b10567600943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.055586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b567d-9072-48ec-a9be-b10567600943 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.055861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b567d-9072-48ec-a9be-b10567600943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.056130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b567d-9072-48ec-a9be-b10567600943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.056833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b567d-9072-48ec-a9be-b10567600943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.058507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b567d-9072-48ec-a9be-b10567600943 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.071249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 930/1886] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:05.079524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.090426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fff8b8-13da-4f84-a003-d6d840c83116 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.095341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.095995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.096268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.096916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] PATH_INFO: `/v3/projects/47b8cf11e60741e29dfec5f97c462eff/users/c51fbcb1f2714d2c825266ec673348a0/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.111901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b8cf11e60741e29dfec5f97c462eff, user_id=c51fbcb1f2714d2c825266ec673348a0, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.114906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.173575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 957/1887] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:05 2022] PUT /identity/v3/projects/47b8cf11e60741e29dfec5f97c462eff/users/c51fbcb1f2714d2c825266ec673348a0/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:05.178684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.178973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.179271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.198793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:05.200155 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:05.256433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.258272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:05.264133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 931/1888] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:05.273744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.275344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:05.277219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db8928-f4ce-407d-a921-3ec14d91bfb1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.283735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.284710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.284972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.285344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] PATH_INFO: `/v3/projects/d2eb08b7b586486e8ff815c65419c1f4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.286820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2eb08b7b586486e8ff815c65419c1f4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.289597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.339140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b018c8-493d-4c74-95c6-8653dd5dca6e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.343571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3c12f-1f42-4072-812a-fd673fe988f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.343828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3c12f-1f42-4072-812a-fd673fe988f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.343983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3c12f-1f42-4072-812a-fd673fe988f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.344273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3c12f-1f42-4072-812a-fd673fe988f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.358701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3c12f-1f42-4072-812a-fd673fe988f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.360173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3c12f-1f42-4072-812a-fd673fe988f0 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.370024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 932/1889] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:05.431164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2eb08b7b586486e8ff815c65419c1f4', 'request_id': 'req-e1711b5a-3263-4ca6-8ec9-7b3abb834660'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:05.432315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2eb08b7b586486e8ff815c65419c1f4', 'request_id': 'req-e1711b5a-3263-4ca6-8ec9-7b3abb834660'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:05.505262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2eb08b7b586486e8ff815c65419c1f4', 'request_id': 'req-e1711b5a-3263-4ca6-8ec9-7b3abb834660'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:05.517627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] The token cache is being invalidate because project d2eb08b7b586486e8ff815c65419c1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:05.517919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:05.518589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e1711b5a-3263-4ca6-8ec9-7b3abb834660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:05.520256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 958/1890] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:05 2022] DELETE /identity/v3/projects/d2eb08b7b586486e8ff815c65419c1f4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:05.525253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.526170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.526738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.547703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:05.548904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:05.555505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:05.610498 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 933/1891] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:05.618611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.618965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.619305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.640009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:05.641279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:05.649699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 959/1892] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:05.654419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.664885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb6de-0d38-4478-ad90-0df78a833486 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.669654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.670847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.671136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.671359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.672006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:05.672302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.673975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.686803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 934/1893] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:41:05.691330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.703268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4979d27-349c-4b5a-b6c4-2d49a9280a26 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.708160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.708890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.709209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.709610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.710408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.712394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.805094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 960/1894] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:05 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:05.808390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.819635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49831ae7-04cb-4345-ae0c-51d0cadc3a14 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:05.880314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:05.880978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:05.881283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:05.881546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:05.882182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:05.884129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:05.970620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 935/1895] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:05 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:05.979993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:05.992894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c2522-3ac1-4f66-8731-bf895c3aad77 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.002448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.003321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.004074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.004505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.005593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.008041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.026303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 961/1896] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:06.027630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.049890 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277ae5-7522-4559-856c-9c4b58099953 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.055088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e926a-58ed-4d50-923d-9cab3b5022af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.055711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e926a-58ed-4d50-923d-9cab3b5022af admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.055994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e926a-58ed-4d50-923d-9cab3b5022af admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.056269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e926a-58ed-4d50-923d-9cab3b5022af admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.057238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e926a-58ed-4d50-923d-9cab3b5022af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.058898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e926a-58ed-4d50-923d-9cab3b5022af admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.072289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 936/1897] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:06.078318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.088846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aab6c4-1597-458f-a52d-2f855e889fe9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.094387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.095056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.095367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.095646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] PATH_INFO: `/v3/projects/a7d64aa0fef4440885c0de7480d4a519/users/874a02b9aa194473ba904db80ed9b4c5/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.112955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d64aa0fef4440885c0de7480d4a519, user_id=874a02b9aa194473ba904db80ed9b4c5, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.115689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.203117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 962/1898] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:06 2022] PUT /identity/v3/projects/a7d64aa0fef4440885c0de7480d4a519/users/874a02b9aa194473ba904db80ed9b4c5/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:06.213544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.213825 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.214099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.245398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] MFA Rules not processed for user `874a02b9aa194473ba904db80ed9b4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:06.246672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:06.321641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 937/1899] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:06.326894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.363690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:06.394245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50428bbe-d3ae-4b3b-8266-c9ffe547ea65 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.399199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.399814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.400115 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.400388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.401057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.403025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.543274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 963/1900] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:06 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:06.558574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.570427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27a65e-e083-488c-9c1b-97e3bf2b825e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.574348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:06.632058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.632669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.632944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.633265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.634019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.636048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.702924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 938/1901] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:06 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:06.704475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.715554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6173968-1ee3-4b2a-a40a-c20d51963498 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.721839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cfb63-fd11-483c-9219-1861430b7801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.723166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cfb63-fd11-483c-9219-1861430b7801 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.723166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cfb63-fd11-483c-9219-1861430b7801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.723166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cfb63-fd11-483c-9219-1861430b7801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.723878 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cfb63-fd11-483c-9219-1861430b7801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.725670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cfb63-fd11-483c-9219-1861430b7801 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.745461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 964/1902] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:06.746811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.749348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.749688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.750000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.758953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183db513-4bb4-4eee-81a7-9c6de58750be admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.763984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.764652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.765082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.765445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] PATH_INFO: `/v3/projects/30ac2bd87fc84c5bbc7f27a5ce11bd69/users/f01a666dfb944124b581fabc2144ffa7/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.781317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] MFA Rules not processed for user `c51fbcb1f2714d2c825266ec673348a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:06.782612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:06.787845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ac2bd87fc84c5bbc7f27a5ce11bd69, user_id=f01a666dfb944124b581fabc2144ffa7, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.791386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.853435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 965/1903] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:06.864947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.866334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:06.900516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 939/1904] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:06 2022] PUT /identity/v3/projects/30ac2bd87fc84c5bbc7f27a5ce11bd69/users/f01a666dfb944124b581fabc2144ffa7/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:06.905258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.935064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f13eb-14db-4ab3-bfc0-4f67b0c05a17 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.971387 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091af2e8-49a9-4069-ba1b-6c61c7dbe9f7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:06.976129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.976741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.977036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.977317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:06.978216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:06.980071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:06.991216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 940/1905] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:06.994924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:06.995177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:06.995664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:06.995889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:06.996162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:07.002094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:07.007904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee931fb-c070-4fea-8a07-281a8191d97e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:07.014756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75ec27-7d55-422f-a6cf-97fb71d01442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:07.015528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75ec27-7d55-422f-a6cf-97fb71d01442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:07.015805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75ec27-7d55-422f-a6cf-97fb71d01442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:07.016075 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75ec27-7d55-422f-a6cf-97fb71d01442 admin admin] PATH_INFO: `/v3/projects/30ac2bd87fc84c5bbc7f27a5ce11bd69/users/f01a666dfb944124b581fabc2144ffa7/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:07.018668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75ec27-7d55-422f-a6cf-97fb71d01442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ac2bd87fc84c5bbc7f27a5ce11bd69, user_id=f01a666dfb944124b581fabc2144ffa7, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:07.021769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75ec27-7d55-422f-a6cf-97fb71d01442 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:07.034107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 941/1906] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:06 2022] PUT /identity/v3/projects/30ac2bd87fc84c5bbc7f27a5ce11bd69/users/f01a666dfb944124b581fabc2144ffa7/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:07.053280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:07.053611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:07.054094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:07.077777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:07.079215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:07.090683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] MFA Rules not processed for user `f01a666dfb944124b581fabc2144ffa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:07.091206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 966/1907] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:07.092739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:07.159255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:07.193444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 942/1908] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:07.207222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:07.210016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:07.225136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484649b4-7eb2-4251-b576-fe76ee13188c service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:07.282916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:07.283500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:07.283779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:07.284069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:07.293227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:07.294649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:07.305295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 967/1909] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:07.534989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:07.545949 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd73da-e26c-43dc-ac21-ae550a112ed3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:07.549740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:07.550341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:07.550612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:07.550878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:07.612124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:07.613603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:07.622592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 943/1910] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:07.658220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:07.667385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec43373-4c5c-474a-9f56-d522bf096952 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:07.671827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:07.672818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:07.672818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:07.673051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:07.674690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:07.742087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:07.743627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf4595-a492-433e-96ff-38d0c939f759 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:07.754430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 968/1911] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:08.054432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:08.056478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:08.128530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c37c-9e5f-4665-b4e8-7ad62f2bdce9 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:08.133962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:08.134797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:08.134797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:08.135104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] PATH_INFO: `/v3/users/64092af98e61463784ca2c7f359d8b90` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:08.136147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64092af98e61463784ca2c7f359d8b90)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:08.138165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:08.307036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64092af98e61463784ca2c7f359d8b90'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:08.319909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64092af98e61463784ca2c7f359d8b90'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:08.367754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64092af98e61463784ca2c7f359d8b90'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:08.368530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64092af98e61463784ca2c7f359d8b90'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:08.369917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64092af98e61463784ca2c7f359d8b90'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:08.382622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:08.391220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 944/1912] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:08 2022] DELETE /identity/v3/users/64092af98e61463784ca2c7f359d8b90 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:08.395586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:08.395951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:08.396293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:08.416722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:08.418121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:08.471668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:08.491226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 969/1913] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:08.501714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:08.503141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:08.576814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6671f0-98b2-4713-9930-c28f9d7ef4e8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:08.576814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661391-33d4-4a07-b049-cf52e6e1dfe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:08.576814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661391-33d4-4a07-b049-cf52e6e1dfe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:08.576814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661391-33d4-4a07-b049-cf52e6e1dfe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:08.577984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661391-33d4-4a07-b049-cf52e6e1dfe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:08.586758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661391-33d4-4a07-b049-cf52e6e1dfe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:08.588241 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661391-33d4-4a07-b049-cf52e6e1dfe1 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:08.598884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 945/1914] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:08.990569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:08.991142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:09.004951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03142a0c-79ee-4aa1-9a4a-3ace0d901c81 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.010597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.011298 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.013438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.013438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] PATH_INFO: `/v3/projects/dd46529250d64c05a4f869388c40cc57` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.013438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd46529250d64c05a4f869388c40cc57)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.016080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.029291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374b878-6afe-4534-aeda-f12e644594f2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.029760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374b878-6afe-4534-aeda-f12e644594f2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.029895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374b878-6afe-4534-aeda-f12e644594f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.049871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8374b878-6afe-4534-aeda-f12e644594f2 None None] MFA Rules not processed for user `c51fbcb1f2714d2c825266ec673348a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:09.051116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8374b878-6afe-4534-aeda-f12e644594f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:09.113222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 946/1915] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:09.113745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd46529250d64c05a4f869388c40cc57', 'request_id': 'req-3c172d94-4e48-480b-b500-afaff739829e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:09.125612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd46529250d64c05a4f869388c40cc57', 'request_id': 'req-3c172d94-4e48-480b-b500-afaff739829e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:09.191239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd46529250d64c05a4f869388c40cc57', 'request_id': 'req-3c172d94-4e48-480b-b500-afaff739829e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:09.206010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] The token cache is being invalidate because project dd46529250d64c05a4f869388c40cc57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:09.206676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:09.207303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3c172d94-4e48-480b-b500-afaff739829e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:09.209969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 970/1916] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:08 2022] DELETE /identity/v3/projects/dd46529250d64c05a4f869388c40cc57 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:09.215081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.215367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.215645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.238086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:09.239326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:09.318543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 947/1917] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:09.326785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.327540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.327847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.349846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:09.351021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:09.360031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 971/1918] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:09.364480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:09.374852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7697b-5533-421f-95e1-f87c5ede7b69 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.382092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.382663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.384912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.384912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.384912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:09.384912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.385908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.398771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 948/1919] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:41:09.404123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:09.421776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8f744-5251-47d9-a64e-83b72e3eaeba None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.426801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.427414 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.427690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.427957 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.428594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.430526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.528099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 972/1920] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:09 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:09.529366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:09.541395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afa33c-ab82-43de-8e46-14515dbf0da4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.596095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:09.628899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.629602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.629881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.630153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.631454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.632711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.729548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 949/1921] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:09 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:09.734915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:09.756128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62716e62-51ab-42dd-b170-54279132e956 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.760845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.761514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.761797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.762232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.762912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.764593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.778330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 973/1922] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:09.786587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:09.790919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cd509-721e-4612-bbe3-9b052ffe835d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.796455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a5c78-28d6-46d1-a4e2-b86304e4b244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.797148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5c78-28d6-46d1-a4e2-b86304e4b244 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.797432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5c78-28d6-46d1-a4e2-b86304e4b244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.797702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5c78-28d6-46d1-a4e2-b86304e4b244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.798348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a5c78-28d6-46d1-a4e2-b86304e4b244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.799955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a5c78-28d6-46d1-a4e2-b86304e4b244 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.813357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 950/1923] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:09.823681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:09.834297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78c979-4486-43eb-b0af-e12af0681984 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:09.838745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:09.839420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.839694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.839970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] PATH_INFO: `/v3/projects/e60a52ce75b94198ac6abb7727e7072d/users/277cc06a6df04ca4ab453de57ee84279/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.857997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60a52ce75b94198ac6abb7727e7072d, user_id=277cc06a6df04ca4ab453de57ee84279, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:09.861278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:09.947949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 974/1924] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:09 2022] PUT /identity/v3/projects/e60a52ce75b94198ac6abb7727e7072d/users/277cc06a6df04ca4ab453de57ee84279/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:09.949424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7223e9-41db-4c08-87f8-3e672eebd72a None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:09.949686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7223e9-41db-4c08-87f8-3e672eebd72a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:09.949954 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7223e9-41db-4c08-87f8-3e672eebd72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:09.972437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-de7223e9-41db-4c08-87f8-3e672eebd72a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:09.973680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-de7223e9-41db-4c08-87f8-3e672eebd72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:10.036688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 951/1925] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:10.047139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:10.049138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:10.108843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eff2dd-fe33-4c10-a99a-130b3464c05b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:10.112828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:10.113408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:10.113594 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:10.113868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:10.123544 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:10.125489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:10.135654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 975/1926] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:16.756046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:16.756592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:16.757241 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:16.793131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] MFA Rules not processed for user `277cc06a6df04ca4ab453de57ee84279`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:16.794532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:16.870827 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 952/1927] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:16.871736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:16.884272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97a88-e7a2-4259-a63f-335718286c32 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:16.895623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:16.896803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:16.897380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:16.897955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:16.898909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:16.901264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:16.906886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:17.026942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 976/1928] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:16 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:17.030723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:17.043165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34b94e-7714-4282-acf2-ff1edbaa7744 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:17.088419 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:17.111122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:17.112040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:17.112377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:17.112713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:17.113486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:17.115782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:17.220780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 953/1929] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:17 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:17.224693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:17.236250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1f7ec-da2e-4bf9-b471-6e10f38ae119 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:17.241034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:17.242457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:17.242738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:17.243334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:17.243990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:17.245619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:17.256547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 977/1930] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:17.262048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:17.272088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb25a-5e6c-46fb-b8e7-469de61b7f5c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:17.276973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:17.277669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:17.277944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:17.278211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] PATH_INFO: `/v3/projects/60b28b9d81b24b5b99dc2152516f4c2c/users/7817e638db8342d697761e7ccaaf1026/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:17.294408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b28b9d81b24b5b99dc2152516f4c2c, user_id=7817e638db8342d697761e7ccaaf1026, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:17.297093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:17.363862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 954/1931] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:17 2022] PUT /identity/v3/projects/60b28b9d81b24b5b99dc2152516f4c2c/users/7817e638db8342d697761e7ccaaf1026/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:17.371453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:17.379788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0aa9e-4931-4744-8935-c907c93627ba admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:17.453671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:17.455058 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:17.455440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:17.455915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:17.456739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:17.458908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:17.470883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 978/1932] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:17.474905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:17.487081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad71d00-3f9f-48f7-b2f3-3925cceab5bd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:17.515259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0065ef7-0b5d-404f-9f1e-d22639831f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:17.516215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0065ef7-0b5d-404f-9f1e-d22639831f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:17.517221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0065ef7-0b5d-404f-9f1e-d22639831f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:17.517613 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0065ef7-0b5d-404f-9f1e-d22639831f1b admin admin] PATH_INFO: `/v3/projects/60b28b9d81b24b5b99dc2152516f4c2c/users/7817e638db8342d697761e7ccaaf1026/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:17.520162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0065ef7-0b5d-404f-9f1e-d22639831f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b28b9d81b24b5b99dc2152516f4c2c, user_id=7817e638db8342d697761e7ccaaf1026, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:17.526047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0065ef7-0b5d-404f-9f1e-d22639831f1b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:17.544558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 955/1933] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:17 2022] PUT /identity/v3/projects/60b28b9d81b24b5b99dc2152516f4c2c/users/7817e638db8342d697761e7ccaaf1026/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:24.493669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:24.498999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:24.527481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930f159-5caa-4c51-b766-b1e6f7ca763e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:24.606211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:24.607102 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:24.607547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:24.608103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] PATH_INFO: `/v3/users/874a02b9aa194473ba904db80ed9b4c5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:24.609953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874a02b9aa194473ba904db80ed9b4c5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:24.612470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:24.808720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:24.809239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:24.809374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:24.824526 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874a02b9aa194473ba904db80ed9b4c5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:24.853660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] MFA Rules not processed for user `7817e638db8342d697761e7ccaaf1026`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:24.855176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:24.877589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874a02b9aa194473ba904db80ed9b4c5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:24.906939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874a02b9aa194473ba904db80ed9b4c5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:24.908352 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874a02b9aa194473ba904db80ed9b4c5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:24.909176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ce91e80c-9305-4a93-a909-a787d3ecb845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874a02b9aa194473ba904db80ed9b4c5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:24.917234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:24.926029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 979/1934] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:24 2022] DELETE /identity/v3/users/874a02b9aa194473ba904db80ed9b4c5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:24.930852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:24.931397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:24.931892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:24.952222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:24.954885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:24.960136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 956/1935] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:24.979375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:24.981270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.058713 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 980/1936] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:25.069088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:25.070935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.071883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec1857-a851-40f7-94d2-bb68b78d8f26 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:25.107715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:25.132382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:25.132382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:25.132382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:25.132382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:25.132742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a35f3-9ea1-4a38-9249-a0b69b4eca96 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:25.132829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.136983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc6008-3073-4778-a7e1-4b4e16caed35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:25.137801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6008-3073-4778-a7e1-4b4e16caed35 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:25.138284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6008-3073-4778-a7e1-4b4e16caed35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:25.140070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6008-3073-4778-a7e1-4b4e16caed35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:25.150952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc6008-3073-4778-a7e1-4b4e16caed35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:25.152488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc6008-3073-4778-a7e1-4b4e16caed35 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:25.171291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 981/1937] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:25.230852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:25.233443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:25.253604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 957/1938] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:25.600466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:25.600947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:25.601060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:25.620403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:25.621620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.666632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:25.668332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.682513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec047de-e4a4-45a9-adb9-7d2a94e9bd72 service glance] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:25.686461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 982/1939] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:41:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:25.692100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:25.692756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:25.693067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:25.693338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] PATH_INFO: `/v3/projects/a7d64aa0fef4440885c0de7480d4a519` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:25.695273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d64aa0fef4440885c0de7480d4a519)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:25.697325 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:25.736069 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:25.737685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.793296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:25.823749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57da047-55c6-4d44-a540-a5082e8b036b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:25.826129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d64aa0fef4440885c0de7480d4a519', 'request_id': 'req-18b93d12-37ec-4434-9124-6735f09d1b88'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:25.832575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d64aa0fef4440885c0de7480d4a519', 'request_id': 'req-18b93d12-37ec-4434-9124-6735f09d1b88'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:25.838584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d64aa0fef4440885c0de7480d4a519', 'request_id': 'req-18b93d12-37ec-4434-9124-6735f09d1b88'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:25.883823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:25.884433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:25.884709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:25.884980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:25.927081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] The token cache is being invalidate because project a7d64aa0fef4440885c0de7480d4a519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:25.927369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:25.928010 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:25.929587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 958/1940] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:25 2022] DELETE /identity/v3/projects/a7d64aa0fef4440885c0de7480d4a519 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:25.933515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:25.934798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:25.944821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:25.946436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:25.959457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 983/1941] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:41:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 14:41:26.026930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b93d12-37ec-4434-9124-6735f09d1b88 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:26.031779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:26.032499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:26.032775 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:26.033083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] PATH_INFO: `/v3/users/f01a666dfb944124b581fabc2144ffa7` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:26.034651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01a666dfb944124b581fabc2144ffa7)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:26.037310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:26.056364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:26.260684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01a666dfb944124b581fabc2144ffa7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:26.284130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01a666dfb944124b581fabc2144ffa7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:26.717684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01a666dfb944124b581fabc2144ffa7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:26.718314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01a666dfb944124b581fabc2144ffa7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:26.719011 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-81ce64a4-56ad-4d0c-bbbe-53fb40c37e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01a666dfb944124b581fabc2144ffa7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:26.742587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 959/1942] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:25 2022] DELETE /identity/v3/users/f01a666dfb944124b581fabc2144ffa7 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:27.622542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:27.644101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1feff8-d845-4bdf-b31e-f2b235cf30a2 service swift] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:27.737826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:27.738473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:27.738749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:27.739042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] PATH_INFO: `/v3/projects/30ac2bd87fc84c5bbc7f27a5ce11bd69` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:27.740145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ac2bd87fc84c5bbc7f27a5ce11bd69)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:27.742627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:27.777171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:27.881177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ac2bd87fc84c5bbc7f27a5ce11bd69', 'request_id': 'req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:27.903174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ac2bd87fc84c5bbc7f27a5ce11bd69', 'request_id': 'req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:27.985983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ac2bd87fc84c5bbc7f27a5ce11bd69', 'request_id': 'req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:28.003970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] The token cache is being invalidate because project 30ac2bd87fc84c5bbc7f27a5ce11bd69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:28.004346 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:28.006913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-e4bc4dd5-9ea1-4b1b-897d-593f26d1ea8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:28.015635 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 984/1943] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:27 2022] DELETE /identity/v3/projects/30ac2bd87fc84c5bbc7f27a5ce11bd69 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:28.017600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.018087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.018087 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.060286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:28.061804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:28.217140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 960/1944] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:28.219263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.221127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.221630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.249669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:28.252092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:28.272956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:28.275972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 985/1945] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:28.282997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb17d4-ed5f-46b3-88e5-b7ba538c63ed None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:28.294020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:28.294645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.294919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.295185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.295833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:28.296131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:28.297811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:28.324231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 961/1946] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:41:28.325444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:28.339966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2dc13-74a5-4780-97a9-f4da9158346d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:28.347366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:28.347993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.348691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.348691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.349207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:28.351133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:28.487285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 986/1947] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:28 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:28.491242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:28.505754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8ef8-77ae-4604-872c-a5543524326d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:28.629927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:28.630611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.630985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.631264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.631998 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:28.635156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:28.786905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 962/1948] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:28 2022] POST /identity/v3/users => generated 282 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:28.787089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:28.827949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba8e60-0a3c-42e2-898d-92b5342e345e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:28.864318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:28.870647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.870647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.870647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.871128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:28.872765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:28.932572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 987/1949] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:28.936702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:28.954487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cdc0a-8839-4a63-8129-37ee3bf26725 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:28.970678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e040854-9ef1-4beb-97e1-f3ba47e0c071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:28.972356 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040854-9ef1-4beb-97e1-f3ba47e0c071 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:28.972943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040854-9ef1-4beb-97e1-f3ba47e0c071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:28.973249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040854-9ef1-4beb-97e1-f3ba47e0c071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:28.973917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e040854-9ef1-4beb-97e1-f3ba47e0c071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:28.975550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e040854-9ef1-4beb-97e1-f3ba47e0c071 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:28.986770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:29.009283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 963/1950] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:29.013232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:29.030408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b358938-4c9a-482a-9d9d-2c1931a8caeb admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:29.036993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:29.037695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:29.037983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:29.038376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] PATH_INFO: `/v3/projects/517d1ce10caa4b8193d0c7e5e5283fbc/users/d42fea21467a49579c5eae74116d6b79/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:29.058408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517d1ce10caa4b8193d0c7e5e5283fbc, user_id=d42fea21467a49579c5eae74116d6b79, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:29.061235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:29.165347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 988/1951] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:29 2022] PUT /identity/v3/projects/517d1ce10caa4b8193d0c7e5e5283fbc/users/d42fea21467a49579c5eae74116d6b79/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:29.173944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:29.175411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:29.176754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:29.315938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] MFA Rules not processed for user `d42fea21467a49579c5eae74116d6b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:29.318043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:29.514274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 964/1952] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:29.520280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:29.655326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55df3d4-7b70-4d8c-80a8-54f3d184f210 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:29.660810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:29.665967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:29.665967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:29.665967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:29.665967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:29.665967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:29.772073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 989/1953] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:29 2022] POST /identity/v3/projects => generated 363 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:29.776962 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:29.804421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af19c-82af-4ff2-872c-62c939d3d39d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:29.906588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:29.908125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:29.908409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:29.910062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:29.914320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:29.914320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:30.301725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 965/1954] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:29 2022] POST /identity/v3/users => generated 288 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:30.325501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:30.336721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac8f72-7543-4a3a-8f35-738cf85d316c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:30.342555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:30.343263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:30.343486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:30.343728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:30.344392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:30.346310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:30.365094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 990/1955] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:30.367293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:30.378512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0120cf-22c3-4f8f-b07c-a95dbd3e55a6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:30.383150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:30.383819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:30.384095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:30.384361 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] PATH_INFO: `/v3/projects/c218b92c9c9d49828ac7d5897211e8b9/users/35915c65d5fc47e289b8d49dc6c720c3/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:30.400581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c218b92c9c9d49828ac7d5897211e8b9, user_id=35915c65d5fc47e289b8d49dc6c720c3, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:30.403319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:30.480761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 966/1956] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:30 2022] PUT /identity/v3/projects/c218b92c9c9d49828ac7d5897211e8b9/users/35915c65d5fc47e289b8d49dc6c720c3/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:30.486276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:30.497906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586cfa3-6889-4ff2-95b1-a4586ad6193f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:30.530000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:30.647437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83751fad-ae74-438c-be8a-7f37076986a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:30.648089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83751fad-ae74-438c-be8a-7f37076986a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:30.648369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83751fad-ae74-438c-be8a-7f37076986a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:30.648642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83751fad-ae74-438c-be8a-7f37076986a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:30.649348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83751fad-ae74-438c-be8a-7f37076986a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:30.651978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83751fad-ae74-438c-be8a-7f37076986a7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:30.673317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 991/1957] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:30.677396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:30.694503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bff03-8abc-4202-a76f-fa97622a727a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:30.723056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:30.723750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:30.724024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:30.724294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] PATH_INFO: `/v3/projects/c218b92c9c9d49828ac7d5897211e8b9/users/35915c65d5fc47e289b8d49dc6c720c3/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:30.728989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c218b92c9c9d49828ac7d5897211e8b9, user_id=35915c65d5fc47e289b8d49dc6c720c3, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:30.733098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:30.759228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 967/1958] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:30 2022] PUT /identity/v3/projects/c218b92c9c9d49828ac7d5897211e8b9/users/35915c65d5fc47e289b8d49dc6c720c3/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:30.770677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:30.775685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:30.775685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:30.810936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] MFA Rules not processed for user `35915c65d5fc47e289b8d49dc6c720c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:30.812182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:30.971179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 992/1959] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:30.987240 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:30.991804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:31.087432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:31.126649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa953f-6f61-4621-ab14-75e9d496df9b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:31.147742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f863c6-8760-42cc-9c7f-b89522499af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:31.148631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f863c6-8760-42cc-9c7f-b89522499af0 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:31.149118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f863c6-8760-42cc-9c7f-b89522499af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:31.149585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f863c6-8760-42cc-9c7f-b89522499af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:31.178073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f863c6-8760-42cc-9c7f-b89522499af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:31.183928 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f863c6-8760-42cc-9c7f-b89522499af0 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:31.212027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 968/1960] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:32.505808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:32.588255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb68fd-b34e-40be-8137-c41a58c54742 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:32.593986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:32.594172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:32.594488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:32.594850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] PATH_INFO: `/v3/users/d42fea21467a49579c5eae74116d6b79` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:32.596064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42fea21467a49579c5eae74116d6b79)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:32.598172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:32.861764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:32.863609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42fea21467a49579c5eae74116d6b79'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:32.929585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42fea21467a49579c5eae74116d6b79'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:32.944032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42fea21467a49579c5eae74116d6b79'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:32.944745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42fea21467a49579c5eae74116d6b79'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:32.945383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42fea21467a49579c5eae74116d6b79'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:32.963838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 993/1961] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:32 2022] DELETE /identity/v3/users/d42fea21467a49579c5eae74116d6b79 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:32.968378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:32.969503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:32.969503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:32.992205 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:32.993447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:33.065345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 969/1962] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:33.072228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:33.076015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:33.158129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c230b-cd2f-4cfe-b4f6-f92ff826bc7a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:33.162466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:33.163093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:33.163520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:33.163798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:33.176470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:33.177947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:33.188794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 994/1963] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:33.947445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:33.948881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:34.014079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541310d-a391-4acf-8b59-d025f2573a3b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:34.033905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:34.034665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:34.035186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:34.035946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] PATH_INFO: `/v3/projects/517d1ce10caa4b8193d0c7e5e5283fbc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:34.039080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517d1ce10caa4b8193d0c7e5e5283fbc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:34.043060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:34.209327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517d1ce10caa4b8193d0c7e5e5283fbc', 'request_id': 'req-0025ddc2-4728-4533-a2ac-a13ff80216c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:34.217072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517d1ce10caa4b8193d0c7e5e5283fbc', 'request_id': 'req-0025ddc2-4728-4533-a2ac-a13ff80216c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:34.237963 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517d1ce10caa4b8193d0c7e5e5283fbc', 'request_id': 'req-0025ddc2-4728-4533-a2ac-a13ff80216c2'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:34.713508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] The token cache is being invalidate because project 517d1ce10caa4b8193d0c7e5e5283fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:34.713508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:34.713508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:34.713508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 970/1964] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:33 2022] DELETE /identity/v3/projects/517d1ce10caa4b8193d0c7e5e5283fbc => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:34.716228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:34.727795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2dce-cee6-4d1a-bd3d-5500f854b178 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:34.805445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:34.806091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:34.806381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:34.806637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] PATH_INFO: `/v3/users/35915c65d5fc47e289b8d49dc6c720c3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:34.807799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35915c65d5fc47e289b8d49dc6c720c3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:34.809887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:35.033448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35915c65d5fc47e289b8d49dc6c720c3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:35.111116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35915c65d5fc47e289b8d49dc6c720c3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:35.153381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:35.156472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35915c65d5fc47e289b8d49dc6c720c3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:35.158585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35915c65d5fc47e289b8d49dc6c720c3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:35.158585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff528c6d-fbdb-4d7f-8432-412fd7151f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35915c65d5fc47e289b8d49dc6c720c3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:35.203548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 995/1965] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:34 2022] DELETE /identity/v3/users/35915c65d5fc47e289b8d49dc6c720c3 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:35.807840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:35.809544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:35.832190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025ddc2-4728-4533-a2ac-a13ff80216c2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:35.941665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:35.942675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:35.943903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:35.944518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] PATH_INFO: `/v3/projects/c218b92c9c9d49828ac7d5897211e8b9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:35.946709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c218b92c9c9d49828ac7d5897211e8b9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:35.948161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:35.958836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:36.084036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c218b92c9c9d49828ac7d5897211e8b9', 'request_id': 'req-f73bcd4e-42d7-44eb-8939-d85322a22b0a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:36.091424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c218b92c9c9d49828ac7d5897211e8b9', 'request_id': 'req-f73bcd4e-42d7-44eb-8939-d85322a22b0a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:36.102332 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c218b92c9c9d49828ac7d5897211e8b9', 'request_id': 'req-f73bcd4e-42d7-44eb-8939-d85322a22b0a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:36.183368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] The token cache is being invalidate because project c218b92c9c9d49828ac7d5897211e8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:36.183821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:36.187672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f73bcd4e-42d7-44eb-8939-d85322a22b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:36.190295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 971/1966] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:35 2022] DELETE /identity/v3/projects/c218b92c9c9d49828ac7d5897211e8b9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:36.195015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:36.196783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:36.196783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:36.220665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:36.221932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:36.311334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 996/1967] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:36.319547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:36.319834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:36.320148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:36.348269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:36.349561 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:36.363330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 972/1968] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:36.365111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:36.378840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d977-780f-45fa-9d4f-5e0bf9d77f46 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:36.409444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:36.410457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:36.410747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:36.411020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:36.411724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:36.412038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:36.416129 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:36.435326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 997/1969] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:41:36.436530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:36.449303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dab8e-7c10-47b3-9369-55107a572272 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:36.455347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:36.456876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:36.456876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:36.456876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:36.457319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:36.459267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:36.589599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 973/1970] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:36 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:36.593817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:36.622348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbd07c-c725-495d-9f2e-2d848584ff39 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:36.792929 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:36.793604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:36.793884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:36.794153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:36.794816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:36.796785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:36.913665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 998/1971] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:36 2022] POST /identity/v3/users => generated 287 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:36.917468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:36.929964 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bedfc-28f6-47aa-8b12-a9b857db6f7b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:36.936925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:36.937642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:36.938280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:36.938580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:36.939878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:36.942058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:36.958789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 974/1972] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:36.963332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:37.003078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55ae9f-f263-4240-be82-6e5124eb2928 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:37.010145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b1fa4-d305-4556-a4a6-efb17f7562eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:37.014401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b1fa4-d305-4556-a4a6-efb17f7562eb admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:37.014401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b1fa4-d305-4556-a4a6-efb17f7562eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:37.014401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b1fa4-d305-4556-a4a6-efb17f7562eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:37.014401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b1fa4-d305-4556-a4a6-efb17f7562eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:37.014813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b1fa4-d305-4556-a4a6-efb17f7562eb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:37.030348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 999/1973] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:37.031545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:37.045404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a6e69-75cc-4fd2-9e8b-40a85fa05ee7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:37.059035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:37.063585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:37.065478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:37.065478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] PATH_INFO: `/v3/projects/72df3c75450d46568090a0a85980c438/users/b9d04d42bc204cef99fa946d5e381ac0/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:37.105788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72df3c75450d46568090a0a85980c438, user_id=b9d04d42bc204cef99fa946d5e381ac0, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:37.112200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:37.182764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 975/1974] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:37 2022] PUT /identity/v3/projects/72df3c75450d46568090a0a85980c438/users/b9d04d42bc204cef99fa946d5e381ac0/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:37.183689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d1655-cae1-4735-9ab3-34399654ea68 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:37.184308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d1655-cae1-4735-9ab3-34399654ea68 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:37.184850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d1655-cae1-4735-9ab3-34399654ea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:37.224984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-138d1655-cae1-4735-9ab3-34399654ea68 None None] MFA Rules not processed for user `b9d04d42bc204cef99fa946d5e381ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:37.226575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-138d1655-cae1-4735-9ab3-34399654ea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:37.290678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:37.326793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1000/1975] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:37.335911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:37.338459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:37.442307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f79575-0f0e-46a7-a133-b7d411a4aef4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:37.446122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:37.446857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:37.447272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:37.447607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:37.457585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:37.461651 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:37.468788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 976/1976] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:40.135187 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fed4d-7a4c-44e1-b182-334c711fa382 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:40.135783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fed4d-7a4c-44e1-b182-334c711fa382 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:40.135783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fed4d-7a4c-44e1-b182-334c711fa382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:40.162244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-064fed4d-7a4c-44e1-b182-334c711fa382 None None] MFA Rules not processed for user `277cc06a6df04ca4ab453de57ee84279`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:40.163581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-064fed4d-7a4c-44e1-b182-334c711fa382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:40.223192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1001/1977] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:42.929896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:42.969937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:42.994975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dd8f1-790e-4e7e-8863-9d136aecf3c9 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:42.999953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:43.000596 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:43.000874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:43.001208 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] PATH_INFO: `/v3/users/b9d04d42bc204cef99fa946d5e381ac0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:43.002375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d04d42bc204cef99fa946d5e381ac0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:43.004293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:43.179763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d04d42bc204cef99fa946d5e381ac0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:43.195517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d04d42bc204cef99fa946d5e381ac0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:43.256503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d04d42bc204cef99fa946d5e381ac0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:43.257299 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d04d42bc204cef99fa946d5e381ac0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:43.258213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d04d42bc204cef99fa946d5e381ac0'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:43.272897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 977/1978] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:42 2022] DELETE /identity/v3/users/b9d04d42bc204cef99fa946d5e381ac0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:43.277884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:43.278396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:43.278496 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:43.298984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:43.300190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:43.368139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1002/1979] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:43.379367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:43.381030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:43.458248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2532015-5351-4bff-9ef9-c7828b777140 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:43.465278 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:43.466323 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:43.466815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:43.467307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:43.491343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:43.491873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:43.507779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 978/1980] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:43.851541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:43.853156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:43.866427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa3756-4018-466e-9f55-1c409c97c2a1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:43.872323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:43.873450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:43.873450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:43.873633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] PATH_INFO: `/v3/users/ac9c12fecc044219b431e8dc3a822bb8` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:43.875994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9c12fecc044219b431e8dc3a822bb8)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:43.877408 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.058147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.059964 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:44.076129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58aff40-478f-42c8-a7c6-bc4b6a949e58 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.082088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.082663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.082937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.083207 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] PATH_INFO: `/v3/projects/72df3c75450d46568090a0a85980c438` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.084593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72df3c75450d46568090a0a85980c438)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.086529 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.110491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9c12fecc044219b431e8dc3a822bb8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.185785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9c12fecc044219b431e8dc3a822bb8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.198967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9c12fecc044219b431e8dc3a822bb8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.199592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9c12fecc044219b431e8dc3a822bb8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.200190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-03ddc1cc-311f-4cdb-972a-d5b0c3fc0c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9c12fecc044219b431e8dc3a822bb8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.208106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:44.213471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1003/1981] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:43 2022] DELETE /identity/v3/users/ac9c12fecc044219b431e8dc3a822bb8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:44.217963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.217963 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.218167 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.239012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:44.240238 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:44.241339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72df3c75450d46568090a0a85980c438', 'request_id': 'req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.246929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:44.250783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72df3c75450d46568090a0a85980c438', 'request_id': 'req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.259758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72df3c75450d46568090a0a85980c438', 'request_id': 'req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.304945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1004/1982] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:44.315634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.316980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:44.339541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] The token cache is being invalidate because project 72df3c75450d46568090a0a85980c438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:44.344309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.344309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f654d3ca-6cd5-4afc-8105-8cf7d52f6557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:44.344309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 979/1983] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:44 2022] DELETE /identity/v3/projects/72df3c75450d46568090a0a85980c438 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:44.347607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729b104-e159-4514-a08e-4f50bd0913cb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.347909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729b104-e159-4514-a08e-4f50bd0913cb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.348210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729b104-e159-4514-a08e-4f50bd0913cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.368581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f729b104-e159-4514-a08e-4f50bd0913cb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:44.369834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f729b104-e159-4514-a08e-4f50bd0913cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:44.378983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 980/1984] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:44.380160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f83a-b909-46b2-96d7-3a676a2371f7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.384812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.385516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.385802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.386080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.392256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.392536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.392806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.393324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:44.407269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.409570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.415468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:44.416684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:44.420324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1005/1985] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:44.425446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 981/1986] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:44.429676 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.439337 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15c4e0-a442-479b-8f7e-b79a0d79741e service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.444496 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.445305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.445598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.445852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.446495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:44.446811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.448402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.463767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1006/1987] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:41:44.470933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.482838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849d8c9-87c5-4d08-9450-860b9b5a87e0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.490318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.496262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.496262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.496262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.496262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.496262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.616858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 982/1988] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:44 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:44.620812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.631093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57434ae9-6fe2-4850-9b03-893b46a92621 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.708890 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.709607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.709889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.710164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.710807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.712933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.788664 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1007/1989] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:44 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:44.789871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.801321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea50e6-8219-455f-9173-2d588eb2acad admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.806089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.806730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.807011 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.807283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.808831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.809574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.824524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 983/1990] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:44.826012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.837179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babad419-1094-44e8-83b1-e6be0498c722 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.844995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5943-278c-4bc4-9929-a2d116fa68e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.848287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5943-278c-4bc4-9929-a2d116fa68e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.848574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5943-278c-4bc4-9929-a2d116fa68e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.848854 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5943-278c-4bc4-9929-a2d116fa68e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.849616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5943-278c-4bc4-9929-a2d116fa68e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.851444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5943-278c-4bc4-9929-a2d116fa68e9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.867880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1008/1991] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:44.868539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:44.881552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45f86-cf04-43ca-ba5d-70c075d46471 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:44.887405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:44.889360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.889819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.890457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] PATH_INFO: `/v3/projects/40f53da1d98d4d198aa24709c5debd11/users/cadb2d01243643deb9fda9ba898f7ce8/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.915105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f53da1d98d4d198aa24709c5debd11, user_id=cadb2d01243643deb9fda9ba898f7ce8, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:44.917846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:44.992693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:44.992994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:44.993475 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:44.996148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 984/1992] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:44 2022] PUT /identity/v3/projects/40f53da1d98d4d198aa24709c5debd11/users/cadb2d01243643deb9fda9ba898f7ce8/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:45.008098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:45.021983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.026287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:45.037928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] MFA Rules not processed for user `cadb2d01243643deb9fda9ba898f7ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:45.040728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:45.115774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bd1d2-e7bd-49a7-b9cb-a67cc191c648 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.121340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.121986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.122272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.122724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] PATH_INFO: `/v3/projects/d95bb59c3bd34c30b86182774de4df2b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.123885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95bb59c3bd34c30b86182774de4df2b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.126142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.135321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1009/1993] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:45.140011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.152357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bbe5e-563d-4962-b829-8d3e6e709900 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.158205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.159248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.159532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.159804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.160483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.162643 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.238295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95bb59c3bd34c30b86182774de4df2b', 'request_id': 'req-9974cd64-48d1-4520-bba7-8c6d361c4fc7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.244848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95bb59c3bd34c30b86182774de4df2b', 'request_id': 'req-9974cd64-48d1-4520-bba7-8c6d361c4fc7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.253040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1010/1994] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:45 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:45.262437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95bb59c3bd34c30b86182774de4df2b', 'request_id': 'req-9974cd64-48d1-4520-bba7-8c6d361c4fc7'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.262590 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.320614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80cff-b71d-4d3f-8d13-066561ff0e1a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.325939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.326578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.326856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.327128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.327771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.329860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.347932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:45.355181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] The token cache is being invalidate because project d95bb59c3bd34c30b86182774de4df2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:45.355181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.355181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.358631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 985/1995] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:45 2022] DELETE /identity/v3/projects/d95bb59c3bd34c30b86182774de4df2b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:45.360318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.362089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:45.436772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974cd64-48d1-4520-bba7-8c6d361c4fc7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.444640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1011/1996] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:45 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:45.445876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.446547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.446826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.451888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] PATH_INFO: `/v3/users/36ad43b19dc04639ad38fe2e9e2d05f4` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.451888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ad43b19dc04639ad38fe2e9e2d05f4)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.451888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.455331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.458950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:45.474018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278469f-4d37-4272-8548-930df7ece21e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.479941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.486533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.486533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.486533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.486533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.486533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.498081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1012/1997] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:45.499348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.509280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a16c86-d381-4790-9b87-adffec834126 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.514179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.514780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.515059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.515326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.515959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.517672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.528558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1013/1998] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:45.533836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.545012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df755ba-edbd-4d4f-919f-12f3ddab0640 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:45.550541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567402-8f93-4ef1-936b-49fba9b6b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:45.551190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567402-8f93-4ef1-936b-49fba9b6b8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.551462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567402-8f93-4ef1-936b-49fba9b6b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.551730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567402-8f93-4ef1-936b-49fba9b6b8e8 admin admin] PATH_INFO: `/v3/projects/2b1a0fd84fa24a48b8365752784174e1/users/91aaa0e766064dca82fa13df42dad280/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.567111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567402-8f93-4ef1-936b-49fba9b6b8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1a0fd84fa24a48b8365752784174e1, user_id=91aaa0e766064dca82fa13df42dad280, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:45.569818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567402-8f93-4ef1-936b-49fba9b6b8e8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:45.690760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1014/1999] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:45 2022] PUT /identity/v3/projects/2b1a0fd84fa24a48b8365752784174e1/users/91aaa0e766064dca82fa13df42dad280/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:45.702596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:45.702925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:45.703138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:45.731423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ad43b19dc04639ad38fe2e9e2d05f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.746903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ad43b19dc04639ad38fe2e9e2d05f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.748809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] MFA Rules not processed for user `91aaa0e766064dca82fa13df42dad280`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:45.750138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:45.756330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:45.831819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ad43b19dc04639ad38fe2e9e2d05f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.835944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ad43b19dc04639ad38fe2e9e2d05f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.835944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ad43b19dc04639ad38fe2e9e2d05f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:45.836153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1015/2000] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:45.840399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:45.850159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 986/2001] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:45 2022] DELETE /identity/v3/users/36ad43b19dc04639ad38fe2e9e2d05f4 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:45.945307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7fa71-f77b-43ce-a38f-948191a75f74 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.034403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.035038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.035468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.035795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.037147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.039119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.133723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1016/2002] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:45 2022] POST /identity/v3/projects => generated 383 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:46.138492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.142946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:46.162731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ff80-2276-4bbe-8871-a5c2cd102651 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.242669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.243491 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.243708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.244032 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.245247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.248241 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.374267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 987/2003] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:46 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:41:46.378564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.388701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e442c-8b20-4cdb-9d53-0fbe1d11fe0d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.406220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed398f75-dbe5-4c2b-82cf-a8dc55156105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.406220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed398f75-dbe5-4c2b-82cf-a8dc55156105 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.406220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed398f75-dbe5-4c2b-82cf-a8dc55156105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.406220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed398f75-dbe5-4c2b-82cf-a8dc55156105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.406766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed398f75-dbe5-4c2b-82cf-a8dc55156105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.406766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed398f75-dbe5-4c2b-82cf-a8dc55156105 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.409244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.412660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:46.420173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1017/2004] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:46.424514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.424514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.424514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.426807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb27d9c-8236-4b0e-9568-af96aa7f6409 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.433228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.433228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.433228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.433228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] PATH_INFO: `/v3/projects/f20120e071fe41249aefbaf1e9c92e5d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.434226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20120e071fe41249aefbaf1e9c92e5d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.436250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.439174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:46.451809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] MFA Rules not processed for user `921ba33e2e0740278b7b6fb4af248363`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:46.453350 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:46.487065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:46.520279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1018/2005] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:46.521176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.533583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3fb67-0340-4039-a5f1-1de981acfb00 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.549761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20120e071fe41249aefbaf1e9c92e5d', 'request_id': 'req-3e2e1959-d78a-4c33-ad98-af33a134a008'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:46.566202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20120e071fe41249aefbaf1e9c92e5d', 'request_id': 'req-3e2e1959-d78a-4c33-ad98-af33a134a008'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:46.573191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20120e071fe41249aefbaf1e9c92e5d', 'request_id': 'req-3e2e1959-d78a-4c33-ad98-af33a134a008'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:46.612531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.613516 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.613938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.614546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] PATH_INFO: `/v3/projects/251a1251c1664c7c9883978abe0c8751/users/94ba8c70483d47a38307e53609a807bc/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.635541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=251a1251c1664c7c9883978abe0c8751, user_id=94ba8c70483d47a38307e53609a807bc, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.638351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.670397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] The token cache is being invalidate because project f20120e071fe41249aefbaf1e9c92e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:46.670637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:46.671267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3e2e1959-d78a-4c33-ad98-af33a134a008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:46.672801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 988/2006] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:46 2022] DELETE /identity/v3/projects/f20120e071fe41249aefbaf1e9c92e5d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:46.678786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb894a5-36d1-4efe-bb5f-1abea04f70d4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.679100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb894a5-36d1-4efe-bb5f-1abea04f70d4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.679375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb894a5-36d1-4efe-bb5f-1abea04f70d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.700249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9bb894a5-36d1-4efe-bb5f-1abea04f70d4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:46.701503 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9bb894a5-36d1-4efe-bb5f-1abea04f70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:46.710937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 989/2007] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:46.719007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.719269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.719540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.742341 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:46.743573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:46.753765 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 990/2008] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:46.758221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.768358 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d829fe-40b1-42c4-9c86-cb7267a50ec6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.773519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.774126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.776397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.776397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.776397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:41:46.776397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.777438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.791592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 991/2009] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:41:46.793886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1019/2010] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:46 2022] PUT /identity/v3/projects/251a1251c1664c7c9883978abe0c8751/users/94ba8c70483d47a38307e53609a807bc/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:46.798333 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.799663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.802914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:46.811979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec9828-16cb-4952-a5d9-69fad9d74f10 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.891326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdec-eb47-4d67-8ced-f93b8be970ff admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.900209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.900562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.902314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.902314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.902314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.903548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.923476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1020/2011] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:46.923700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.923700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.923700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.923700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.924194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.924558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:46.926161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.939279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f682-336c-45ea-bb50-00f671497563 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:46.945181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e43aca-4a7d-4366-bd5b-8c1edc6eddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:46.945865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e43aca-4a7d-4366-bd5b-8c1edc6eddcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.946146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e43aca-4a7d-4366-bd5b-8c1edc6eddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.946608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e43aca-4a7d-4366-bd5b-8c1edc6eddcd admin admin] PATH_INFO: `/v3/projects/251a1251c1664c7c9883978abe0c8751/users/94ba8c70483d47a38307e53609a807bc/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:46.949274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e43aca-4a7d-4366-bd5b-8c1edc6eddcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=251a1251c1664c7c9883978abe0c8751, user_id=94ba8c70483d47a38307e53609a807bc, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:46.952237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e43aca-4a7d-4366-bd5b-8c1edc6eddcd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:46.972566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1021/2012] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:46 2022] PUT /identity/v3/projects/251a1251c1664c7c9883978abe0c8751/users/94ba8c70483d47a38307e53609a807bc/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:46.982969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:46.983258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:46.983537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.021586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] MFA Rules not processed for user `94ba8c70483d47a38307e53609a807bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:47.026747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:47.028919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 992/2013] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:46 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:47.031151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:47.073829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:47.136143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:47.144240 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1022/2014] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:47.153070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5b5f0-10e7-4585-8f3c-4ff5ae4c9050 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:47.156391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:47.159484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:47.163501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:47.164412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.164864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.166067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.166588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:47.168665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:47.240287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02667-98f8-4689-841d-3d0802319c29 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:47.243948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:47.244561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.244836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.245137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.246910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:47.311682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 993/2015] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:47 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:47.311884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:47.312647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:47.312945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:47.324468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6c27-b2cb-46b6-8a04-4783daf6e54b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:47.325200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1023/2016] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:47.342281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:47.342918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.343197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.343477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.344257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:47.346633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:47.358398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 994/2017] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:41:47.363637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:47.405333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0f693-364e-431f-8066-746e76499157 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:47.408539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841566fb-06bf-49e3-8495-f234bbd4a52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:47.409407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841566fb-06bf-49e3-8495-f234bbd4a52d admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.409850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841566fb-06bf-49e3-8495-f234bbd4a52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.410268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841566fb-06bf-49e3-8495-f234bbd4a52d admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.412166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841566fb-06bf-49e3-8495-f234bbd4a52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:47.414189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841566fb-06bf-49e3-8495-f234bbd4a52d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:47.450654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1024/2018] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:47.454749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:47.471007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee88d-e326-46f0-8bea-dea00460878a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:47.478730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:47.478730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.478730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.478730 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] PATH_INFO: `/v3/projects/9ca38fcd803c4239bed7c05a5b6a9244/users/8894b1c5b6d448f2aa8dbd4ca2f8f83f/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.498230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca38fcd803c4239bed7c05a5b6a9244, user_id=8894b1c5b6d448f2aa8dbd4ca2f8f83f, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:47.501325 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:47.594601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 995/2019] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:47 2022] PUT /identity/v3/projects/9ca38fcd803c4239bed7c05a5b6a9244/users/8894b1c5b6d448f2aa8dbd4ca2f8f83f/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:47.603938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26223c30-7fbb-4ce9-9a08-4c2aefc11f03 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.604430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26223c30-7fbb-4ce9-9a08-4c2aefc11f03 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.604838 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26223c30-7fbb-4ce9-9a08-4c2aefc11f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.628940 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-26223c30-7fbb-4ce9-9a08-4c2aefc11f03 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:47.630333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-26223c30-7fbb-4ce9-9a08-4c2aefc11f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:47.732562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1025/2020] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:47.744869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:47.746626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:47.811038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286acc3-5749-4123-a2bf-0edfc4611f82 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:47.815755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:47.817110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:47.817110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:47.817110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:47.827007 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:47.828441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:47.849241 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 996/2021] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:49.761710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3ac4e-5234-4a4f-b8f9-01fe31f68491 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:49.762182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3ac4e-5234-4a4f-b8f9-01fe31f68491 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:49.768644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3ac4e-5234-4a4f-b8f9-01fe31f68491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:49.768782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:49.799995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d1d3ac4e-5234-4a4f-b8f9-01fe31f68491 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:49.801778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d1d3ac4e-5234-4a4f-b8f9-01fe31f68491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:49.807575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:49.913069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0684baf-c3c1-4522-9ee1-2a0746e85beb service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:49.920284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:49.920284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:49.920284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:49.920284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:49.934880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:49.965778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1026/2022] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:41:49.967327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:49.967879 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:49.968400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:50.009668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] MFA Rules not processed for user `8894b1c5b6d448f2aa8dbd4ca2f8f83f`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:50.010997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:50.081770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:50.085424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:50.107298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 997/2023] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:41:50.112397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:50.114523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:50.137234 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:50.146678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1027/2024] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:50.150722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:50.181114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f76af4-8f09-43bf-bd0b-038dc0e08c4e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:50.192702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:50.193811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:50.195182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:50.195758 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:50.196875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:50.202417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:50.338359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1028/2025] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:41:50 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:50.339520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:50.378284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b85eb-a5ca-4bb2-b5ac-97f017eafa4c service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:50.515183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:50.515855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:50.516098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:50.516391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:50.545273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0327d-e0a3-4038-920c-645e54b60a83 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:50.573855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:50.576775 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:50.578705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:50.580101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:50.585328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:50.600403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:50.672510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:50.676618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:50.698000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 998/2026] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:41:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:41:50.792608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1029/2027] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:41:50 2022] POST /identity/v3/users => generated 293 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:41:50.794125 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:50.815322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0091eb-5a4e-4d05-b742-0f2ef2a55a1c service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:50.829714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:50.830652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:50.831086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:50.831505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:50.833258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:50.835943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:50.853268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 999/2028] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:50.856547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:50.892543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6b79a-4e56-4b78-a209-c79bf1c36928 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:50.940545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:50.941906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:50.942366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:50.942813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] PATH_INFO: `/v3/projects/57ea3f28b9714305af35c7e5badf8888/users/29ed1efc55254ca2bd1936153bba673f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:50.954587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:50.965711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ea3f28b9714305af35c7e5badf8888, user_id=29ed1efc55254ca2bd1936153bba673f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:50.971293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:51.128292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1030/2029] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:50 2022] PUT /identity/v3/projects/57ea3f28b9714305af35c7e5badf8888/users/29ed1efc55254ca2bd1936153bba673f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:51.135847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:51.160026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d269214-6d48-4d56-b5e0-a202580a02ac admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:51.206003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:51.270368 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:51.274469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:51.274757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:51.275031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:51.275701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:51.277382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:51.319647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1000/2030] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:41:51.322807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:51.350990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25538de-77a6-46fd-9e98-4161eb725b8a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:51.366760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:51.366760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:51.367221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:51.368277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] PATH_INFO: `/v3/projects/57ea3f28b9714305af35c7e5badf8888/users/29ed1efc55254ca2bd1936153bba673f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:51.374190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ea3f28b9714305af35c7e5badf8888, user_id=29ed1efc55254ca2bd1936153bba673f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:51.381532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:51.413731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1031/2031] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:41:51 2022] PUT /identity/v3/projects/57ea3f28b9714305af35c7e5badf8888/users/29ed1efc55254ca2bd1936153bba673f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:52.403914 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:52.546666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa313811-5eb2-4e3a-b7de-2bf4431e22dd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:52.759841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2432b-8837-4018-b8db-9318b38fe27c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:52.762184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2432b-8837-4018-b8db-9318b38fe27c service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:52.763239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2432b-8837-4018-b8db-9318b38fe27c service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:52.763782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2432b-8837-4018-b8db-9318b38fe27c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:52.841953 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:52.844477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:53.032841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2bd64-7aed-4be4-932b-f2dd870cbb95 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:53.041755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:53.042961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:53.046053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:53.046413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] PATH_INFO: `/v3/users/c854907702fe41a3b89b858bb6da0e8e` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:53.050387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c854907702fe41a3b89b858bb6da0e8e)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:53.056701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:53.062097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2432b-8837-4018-b8db-9318b38fe27c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:53.068172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2432b-8837-4018-b8db-9318b38fe27c service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:53.096192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1001/2032] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:53.274861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c854907702fe41a3b89b858bb6da0e8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:53.340497 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c854907702fe41a3b89b858bb6da0e8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:53.364341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c854907702fe41a3b89b858bb6da0e8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:53.365112 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c854907702fe41a3b89b858bb6da0e8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:53.377094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c854907702fe41a3b89b858bb6da0e8e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:53.417896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1032/2033] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:41:52 2022] DELETE /identity/v3/users/c854907702fe41a3b89b858bb6da0e8e => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:41:53.418106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a9e1-2046-4a7d-961f-b0b5c797ec82 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:53.418106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a9e1-2046-4a7d-961f-b0b5c797ec82 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:53.418106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a9e1-2046-4a7d-961f-b0b5c797ec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:53.474150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-fdf2a9e1-2046-4a7d-961f-b0b5c797ec82 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:53.477970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fdf2a9e1-2046-4a7d-961f-b0b5c797ec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:53.580567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:53.682261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1002/2034] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:53.719639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:53.721683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:53.878021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1009cb-b4f3-438f-8a1b-b8e04280fbf7 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:53.893312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:53.893959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:53.894236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:53.897693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:53.905571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:53.910707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:53.912116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:53.926581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1033/2035] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:53.940191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e4f47-ce8c-482e-8aea-d848ac10efd8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:53.943572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e4f47-ce8c-482e-8aea-d848ac10efd8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:53.943572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e4f47-ce8c-482e-8aea-d848ac10efd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:53.994715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-691e4f47-ce8c-482e-8aea-d848ac10efd8 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:53.996382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-691e4f47-ce8c-482e-8aea-d848ac10efd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:54.132666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1003/2036] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:41:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:54.148162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:54.150044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:54.263982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:54.263982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:54.263982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:54.360003 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] MFA Rules not processed for user `29ed1efc55254ca2bd1936153bba673f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:54.362567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:54.375494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6943-9f49-4583-9f8d-bc46cb786de2 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:54.411333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bce8a9-fa91-4558-9156-bed6541a2429 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:54.411983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bce8a9-fa91-4558-9156-bed6541a2429 service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:54.417377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bce8a9-fa91-4558-9156-bed6541a2429 service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:54.417377 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bce8a9-fa91-4558-9156-bed6541a2429 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:54.422778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bce8a9-fa91-4558-9156-bed6541a2429 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:54.425342 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bce8a9-fa91-4558-9156-bed6541a2429 service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:54.444836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1034/2037] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:41:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:41:54.449047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:54.450331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:54.451156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:54.568842 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] MFA Rules not processed for user `c51fbcb1f2714d2c825266ec673348a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:41:54.571599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:54.599370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1004/2038] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:54.600372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:54.604903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:54.786214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1035/2039] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:41:54.858171 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf61b0-0c5d-4209-8c62-d031d2051db7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:54.870449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:54.877769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:54.877769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:54.877769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:54.884955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:54.960866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:41:55.066027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:55.068528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:55.096574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1005/2040] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:41:56.192366 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:56.196134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:56.230301 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dc81a-3acc-4852-925d-5e987f2dd589 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:56.242559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:56.243188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:56.243474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:56.243740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] PATH_INFO: `/v3/projects/5528efbb3dcf4a3fbc892cb01540ca66` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:56.252541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5528efbb3dcf4a3fbc892cb01540ca66)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:56.254356 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:56.506728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5528efbb3dcf4a3fbc892cb01540ca66', 'request_id': 'req-768a6541-d252-4f39-b133-36a7d8388553'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:56.525548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5528efbb3dcf4a3fbc892cb01540ca66', 'request_id': 'req-768a6541-d252-4f39-b133-36a7d8388553'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:56.620557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5528efbb3dcf4a3fbc892cb01540ca66', 'request_id': 'req-768a6541-d252-4f39-b133-36a7d8388553'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:56.665391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] The token cache is being invalidate because project 5528efbb3dcf4a3fbc892cb01540ca66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:41:56.665391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:56.665391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:41:56.665391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1036/2041] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:41:56 2022] DELETE /identity/v3/projects/5528efbb3dcf4a3fbc892cb01540ca66 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:41:58.067401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:41:58.080740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf95d53-871a-4bb1-8dfd-432b90f8243b service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:41:58.237690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:41:58.238468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:41:58.242446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:41:58.242446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:41:58.246128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:41:58.405414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:41:58.407320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea379099-3e96-4fd7-9460-c58fd49015dc service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:41:58.446715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1006/2042] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:41:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:01.190859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:01.192816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:01.221179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:01.327699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a6541-d252-4f39-b133-36a7d8388553 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:01.350440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:01.350947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:01.351293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:01.351645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] PATH_INFO: `/v3/users/cadb2d01243643deb9fda9ba898f7ce8` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:01.357403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadb2d01243643deb9fda9ba898f7ce8)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:01.359327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:01.679990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadb2d01243643deb9fda9ba898f7ce8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:01.728309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadb2d01243643deb9fda9ba898f7ce8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:01.768132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadb2d01243643deb9fda9ba898f7ce8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:01.770158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadb2d01243643deb9fda9ba898f7ce8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:01.772763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadb2d01243643deb9fda9ba898f7ce8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:01.805224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1037/2043] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:01 2022] DELETE /identity/v3/users/cadb2d01243643deb9fda9ba898f7ce8 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:01.812715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:01.813401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:01.814687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:01.850064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:01.858041 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:02.002493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1007/2044] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:02.012696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:02.016776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:02.251185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32bfd1-f161-4aad-a890-11a9524165b5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:02.272148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:02.272865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:02.275227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:02.275511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:02.310939 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:02.312982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:02.349688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1038/2045] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:03.692636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:03.694717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:03.732062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce7fe2-c6a9-4828-98ee-141683734f8c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:03.754527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:03.755268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:03.755470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:03.755841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] PATH_INFO: `/v3/projects/40f53da1d98d4d198aa24709c5debd11` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:03.757431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f53da1d98d4d198aa24709c5debd11)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:03.759531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:03.784011 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:03.941139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f53da1d98d4d198aa24709c5debd11', 'request_id': 'req-66fe220c-b238-4a67-9cbc-18ad5a39f349'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:03.957793 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f53da1d98d4d198aa24709c5debd11', 'request_id': 'req-66fe220c-b238-4a67-9cbc-18ad5a39f349'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:03.977809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f53da1d98d4d198aa24709c5debd11', 'request_id': 'req-66fe220c-b238-4a67-9cbc-18ad5a39f349'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.080418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] The token cache is being invalidate because project 40f53da1d98d4d198aa24709c5debd11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:04.080418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.088926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.088926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1008/2046] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:03 2022] DELETE /identity/v3/projects/40f53da1d98d4d198aa24709c5debd11 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:04.091339 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:04.104591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3ddb9-c882-44ca-aeca-2c2df1b955e3 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:04.207974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:04.245824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:04.246524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:04.246944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:04.247583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] PATH_INFO: `/v3/users/91aaa0e766064dca82fa13df42dad280` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:04.254223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91aaa0e766064dca82fa13df42dad280)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:04.262557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:04.774617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91aaa0e766064dca82fa13df42dad280'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.855769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91aaa0e766064dca82fa13df42dad280'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.874574 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91aaa0e766064dca82fa13df42dad280'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.875836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91aaa0e766064dca82fa13df42dad280'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.878933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91aaa0e766064dca82fa13df42dad280'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:04.920641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1039/2047] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:04 2022] DELETE /identity/v3/users/91aaa0e766064dca82fa13df42dad280 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:06.284475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:06.288923 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:06.304641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe220c-b238-4a67-9cbc-18ad5a39f349 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:06.488127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:06.489089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:06.489401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:06.489863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] PATH_INFO: `/v3/projects/2b1a0fd84fa24a48b8365752784174e1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:06.491363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b1a0fd84fa24a48b8365752784174e1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:06.493789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:06.638966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b1a0fd84fa24a48b8365752784174e1', 'request_id': 'req-27957aca-a62c-44ba-a26b-07168a0b1bd4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:06.648066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b1a0fd84fa24a48b8365752784174e1', 'request_id': 'req-27957aca-a62c-44ba-a26b-07168a0b1bd4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:06.661586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b1a0fd84fa24a48b8365752784174e1', 'request_id': 'req-27957aca-a62c-44ba-a26b-07168a0b1bd4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:06.755037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] The token cache is being invalidate because project 2b1a0fd84fa24a48b8365752784174e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:06.755393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:06.756182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:06.758691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1009/2048] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:06 2022] DELETE /identity/v3/projects/2b1a0fd84fa24a48b8365752784174e1 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:06.762861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:06.764566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:06.791520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15b9f8-b6f4-44c8-a910-65591fafa9a7 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:06.922564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:06.923525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:06.923855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:06.924100 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] PATH_INFO: `/v3/users/94ba8c70483d47a38307e53609a807bc` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:06.925262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ba8c70483d47a38307e53609a807bc)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:06.927567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:07.174987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ba8c70483d47a38307e53609a807bc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:07.283919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ba8c70483d47a38307e53609a807bc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:07.287610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:07.320755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ba8c70483d47a38307e53609a807bc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:07.320755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ba8c70483d47a38307e53609a807bc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:07.320755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0c56385a-2b4a-4fe5-94cf-0c674c290f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ba8c70483d47a38307e53609a807bc'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:07.328986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1040/2049] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:06 2022] DELETE /identity/v3/users/94ba8c70483d47a38307e53609a807bc => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:08.111314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:08.113090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:08.170857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27957aca-a62c-44ba-a26b-07168a0b1bd4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:08.228672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:08.289777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:08.290756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:08.291139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:08.291514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] PATH_INFO: `/v3/projects/251a1251c1664c7c9883978abe0c8751` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:08.293327 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=251a1251c1664c7c9883978abe0c8751)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:08.296199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:08.483740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251a1251c1664c7c9883978abe0c8751', 'request_id': 'req-087e7609-ba88-41b7-b415-5f4c88b8fd75'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:08.491709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251a1251c1664c7c9883978abe0c8751', 'request_id': 'req-087e7609-ba88-41b7-b415-5f4c88b8fd75'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:08.497496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251a1251c1664c7c9883978abe0c8751', 'request_id': 'req-087e7609-ba88-41b7-b415-5f4c88b8fd75'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:08.649981 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] The token cache is being invalidate because project 251a1251c1664c7c9883978abe0c8751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:08.650565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:08.651411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-087e7609-ba88-41b7-b415-5f4c88b8fd75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:08.653232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1010/2050] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:08 2022] DELETE /identity/v3/projects/251a1251c1664c7c9883978abe0c8751 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:08.660182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:08.660729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:08.660994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:08.683974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:08.685687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:08.792179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1041/2051] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:42:08.797350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:08.797350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:08.797350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:08.827808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:08.830086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:08.842664 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1011/2052] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:08.847177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:08.865798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aa7f3-0328-4a81-9848-cf4f5904be93 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:08.884547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:08.885580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:08.886634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:08.888426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:08.900181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:42:08.901169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:08.903431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:08.946925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1042/2053] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:42:08.967131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:08.967131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ad0fe-20b7-4c30-9228-423c04c3df0f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:09.013351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:09.013351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:09.013351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:09.013351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:09.013865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:09.013865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:09.373057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1012/2054] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:08 2022] POST /identity/v3/projects => generated 373 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:09.376782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:09.393271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dbd91-9657-4867-870f-4d8cf3934c66 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:09.638364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:09.638364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:09.638364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:09.638364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:09.639952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:09.639952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:10.217509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1043/2055] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:09 2022] POST /identity/v3/users => generated 287 bytes in 839 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:10.218668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:10.229801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1a224-0c21-48b0-8ced-f57df97ada87 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:10.234978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:10.236469 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:10.236821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:10.237203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:10.237926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:10.240183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:10.257230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:10.264510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1013/2056] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:10.274322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabffece-e09a-4996-9433-8cc122dfdc93 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:10.281466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83264069-29a4-4da3-9237-cee86dcec1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:10.282095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83264069-29a4-4da3-9237-cee86dcec1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:10.282433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83264069-29a4-4da3-9237-cee86dcec1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:10.282703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83264069-29a4-4da3-9237-cee86dcec1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:10.284290 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83264069-29a4-4da3-9237-cee86dcec1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:10.286311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83264069-29a4-4da3-9237-cee86dcec1a3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:10.305063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1044/2057] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:10.309348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:10.329142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131cfb4-5375-46f3-a228-19ada2ac93e7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:10.353704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:10.357120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:10.357120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:10.357120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] PATH_INFO: `/v3/projects/0e3fe57f040f4d21bca0fddc82fdd5c2/users/a796fe08e3e64a9f85120b551c8b5134/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:10.377938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3fe57f040f4d21bca0fddc82fdd5c2, user_id=a796fe08e3e64a9f85120b551c8b5134, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:10.380711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:10.503577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1014/2058] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:10 2022] PUT /identity/v3/projects/0e3fe57f040f4d21bca0fddc82fdd5c2/users/a796fe08e3e64a9f85120b551c8b5134/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:10.517165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:10.521792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:10.521792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:10.584531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] MFA Rules not processed for user `a796fe08e3e64a9f85120b551c8b5134`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:10.608175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:10.696941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:10.760456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1045/2059] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:42:10.791150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:10.793550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:10.945731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a3fe-a6ec-4267-9566-b4c47c2b98d8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:10.953842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8ea8-20c9-4e5c-a73a-1fbf5ed4d5cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:10.955797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8ea8-20c9-4e5c-a73a-1fbf5ed4d5cf service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:10.958202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8ea8-20c9-4e5c-a73a-1fbf5ed4d5cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:10.958862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8ea8-20c9-4e5c-a73a-1fbf5ed4d5cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:10.968086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:10.971173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8ea8-20c9-4e5c-a73a-1fbf5ed4d5cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:10.972757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8ea8-20c9-4e5c-a73a-1fbf5ed4d5cf service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:10.992291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1015/2060] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:11.059121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:11.227987 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfec161-79b0-4ded-8345-201bdebfcaa3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:11.234774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:11.236809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:11.237323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:11.237761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] PATH_INFO: `/v3/users/a796fe08e3e64a9f85120b551c8b5134` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:11.239195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a796fe08e3e64a9f85120b551c8b5134)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:11.241789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:11.621217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a796fe08e3e64a9f85120b551c8b5134'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:11.754805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a796fe08e3e64a9f85120b551c8b5134'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:11.770361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a796fe08e3e64a9f85120b551c8b5134'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:11.771099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a796fe08e3e64a9f85120b551c8b5134'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:11.771714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a796fe08e3e64a9f85120b551c8b5134'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:11.785498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1046/2061] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:11 2022] DELETE /identity/v3/users/a796fe08e3e64a9f85120b551c8b5134 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:11.790245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:11.790800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:11.790800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:11.812668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:11.814692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:11.903737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1016/2062] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:11.909067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:11.910589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:11.982763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:11.992602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ad2e6-11e9-4d9a-b400-f960ef8ca28b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:11.997494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b4cc3-b8bc-4763-95f1-4bed1cc22b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:11.998222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b4cc3-b8bc-4763-95f1-4bed1cc22b84 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:11.999105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b4cc3-b8bc-4763-95f1-4bed1cc22b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:11.999105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b4cc3-b8bc-4763-95f1-4bed1cc22b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:12.010384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b4cc3-b8bc-4763-95f1-4bed1cc22b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:12.011785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b4cc3-b8bc-4763-95f1-4bed1cc22b84 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:12.041480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1047/2063] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:12.566009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:12.567473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:12.602525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee795f-68b8-4303-bf00-e9fdf9bb89cd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:12.618733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:12.620093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:12.624847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:12.625416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] PATH_INFO: `/v3/projects/0e3fe57f040f4d21bca0fddc82fdd5c2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:12.629815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3fe57f040f4d21bca0fddc82fdd5c2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:12.637354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:12.810179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3fe57f040f4d21bca0fddc82fdd5c2', 'request_id': 'req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:12.818051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3fe57f040f4d21bca0fddc82fdd5c2', 'request_id': 'req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:12.825044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3fe57f040f4d21bca0fddc82fdd5c2', 'request_id': 'req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:12.901867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] The token cache is being invalidate because project 0e3fe57f040f4d21bca0fddc82fdd5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:12.902167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:12.906868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-af3118bb-8373-45bb-a48f-5cae4f5f0b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:12.906868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1017/2064] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:12 2022] DELETE /identity/v3/projects/0e3fe57f040f4d21bca0fddc82fdd5c2 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:12.911017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:12.911577 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:12.911917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:12.951269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:12.953091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:13.029184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1048/2065] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:13.038838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.039412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.044783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.093141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:13.094180 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:13.122864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1018/2066] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:13.127850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:13.138811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3145b-7535-45d7-aaff-7790ca958236 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:13.145656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:13.147050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.147546 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.148023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.148867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:42:13.149413 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:13.151281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:13.170966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1049/2067] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:42:13.175718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:13.187124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea71d1-8515-41f8-8028-8ac81765cdb4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:13.194149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:13.195181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.195671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.196110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.197038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:13.201958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:13.331346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1019/2068] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:13 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:13.331510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:13.339910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876b28-1bb6-4a23-8c83-99ae70fb00d6 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:13.443437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:13.444177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.444477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.444753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.445448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:13.448076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:13.552550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1050/2069] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:13 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:13.557159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:13.584501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715a92-2910-4ce7-b51a-83abb1553e34 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:13.604279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:13.604972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.605395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.605741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.606493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:13.608898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:13.621847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1020/2070] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:13.626915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:13.638711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bd0b2-032d-4778-9d2b-4862f9662588 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:13.645425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3417220-52ae-4448-a8e5-9bee7ed4c058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:13.646177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3417220-52ae-4448-a8e5-9bee7ed4c058 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.646928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3417220-52ae-4448-a8e5-9bee7ed4c058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.646928 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3417220-52ae-4448-a8e5-9bee7ed4c058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.647605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3417220-52ae-4448-a8e5-9bee7ed4c058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:13.649717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3417220-52ae-4448-a8e5-9bee7ed4c058 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:13.664209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1051/2071] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:13.673724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:13.694578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175367-0ae8-4355-b780-d448a1062111 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:13.700166 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:13.701301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.702013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.702530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] PATH_INFO: `/v3/projects/f8e090451c7e4296a0b85c7eeb9002f7/users/2fa6a7aba33d4abd9389ecd2e3fa0906/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.724360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e090451c7e4296a0b85c7eeb9002f7, user_id=2fa6a7aba33d4abd9389ecd2e3fa0906, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:13.728834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:13.842759 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1021/2072] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:13 2022] PUT /identity/v3/projects/f8e090451c7e4296a0b85c7eeb9002f7/users/2fa6a7aba33d4abd9389ecd2e3fa0906/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:13.854519 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ebc47-d177-4388-8bf8-b5ed9e827173 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:13.854806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ebc47-d177-4388-8bf8-b5ed9e827173 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:13.855077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ebc47-d177-4388-8bf8-b5ed9e827173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:13.892512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-696ebc47-d177-4388-8bf8-b5ed9e827173 None None] MFA Rules not processed for user `2fa6a7aba33d4abd9389ecd2e3fa0906`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:13.894219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-696ebc47-d177-4388-8bf8-b5ed9e827173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:13.967764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:13.990506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1052/2073] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:14.003344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:14.005200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:14.031595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:14.073892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca619bec-18af-475e-bc0f-0e14bbaf11bd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:14.077960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:14.078586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:14.078988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:14.079979 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:14.090375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:14.091840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:14.105621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1022/2074] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:16.069572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:16.070050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:16.070133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:16.094051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] MFA Rules not processed for user `2fa6a7aba33d4abd9389ecd2e3fa0906`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:16.095488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:16.103010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1053/2075] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:18.993230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:18.995448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:19.104803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742cfd0-9c4c-41af-a7d2-70f5fa743ef3 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:19.104803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:19.105284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:19.105564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:19.105836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] PATH_INFO: `/v3/users/8894b1c5b6d448f2aa8dbd4ca2f8f83f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:19.107402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8894b1c5b6d448f2aa8dbd4ca2f8f83f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:19.109628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:19.322919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8894b1c5b6d448f2aa8dbd4ca2f8f83f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:19.340321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8894b1c5b6d448f2aa8dbd4ca2f8f83f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:19.431985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8894b1c5b6d448f2aa8dbd4ca2f8f83f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:19.432602 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8894b1c5b6d448f2aa8dbd4ca2f8f83f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:19.433279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-2b4aa554-ea0a-426e-a82d-f2347d4edbc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8894b1c5b6d448f2aa8dbd4ca2f8f83f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:19.453372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1023/2076] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:18 2022] DELETE /identity/v3/users/8894b1c5b6d448f2aa8dbd4ca2f8f83f => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:19.751409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:19.773665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26394633-42fa-4e69-829d-de1e4833b493 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:19.851681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:19.851681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:19.851681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:19.851681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] PATH_INFO: `/v3/projects/9ca38fcd803c4239bed7c05a5b6a9244` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:19.852232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca38fcd803c4239bed7c05a5b6a9244)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:19.852232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:19.995795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca38fcd803c4239bed7c05a5b6a9244', 'request_id': 'req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.007460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca38fcd803c4239bed7c05a5b6a9244', 'request_id': 'req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.083424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca38fcd803c4239bed7c05a5b6a9244', 'request_id': 'req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.099133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] The token cache is being invalidate because project 9ca38fcd803c4239bed7c05a5b6a9244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:20.099440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.100094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.101624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1054/2077] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:19 2022] DELETE /identity/v3/projects/9ca38fcd803c4239bed7c05a5b6a9244 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:20.123091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdca652-fffd-446c-896a-55f25b0130dc None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.123566 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdca652-fffd-446c-896a-55f25b0130dc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.123808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:20.123930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdca652-fffd-446c-896a-55f25b0130dc None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.125564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1024/2078] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:42:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:42:20.128176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:20.132103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.132666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.132898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.156489 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:20.158621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:20.235772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e6c38-16b2-4b6d-b1ad-e1ac37b19d80 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:20.237817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:20.243575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:20.244609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.245111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.245623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] PATH_INFO: `/v3/users/29ed1efc55254ca2bd1936153bba673f` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.247199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ed1efc55254ca2bd1936153bba673f)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:20.250797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:20.277886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:20.322817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1025/2079] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:42:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:20.336550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:20.338619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:20.410744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa98dc-a2a3-4ada-80f4-af444ba7ebf5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:20.414490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede23a77-38f0-4476-80b0-82aa27021fd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '287fbe09913b454c9fcd58febbca6b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:20.415092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23a77-38f0-4476-80b0-82aa27021fd4 service glance] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.415367 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23a77-38f0-4476-80b0-82aa27021fd4 service glance] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.415633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23a77-38f0-4476-80b0-82aa27021fd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.427048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede23a77-38f0-4476-80b0-82aa27021fd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:20.428499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede23a77-38f0-4476-80b0-82aa27021fd4 service glance] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:20.439743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1026/2080] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:20.505551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ed1efc55254ca2bd1936153bba673f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.647831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ed1efc55254ca2bd1936153bba673f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.665632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ed1efc55254ca2bd1936153bba673f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.666376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ed1efc55254ca2bd1936153bba673f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.666985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-87b9808e-a01d-47a8-8d40-0e3a35ad9983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ed1efc55254ca2bd1936153bba673f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:20.684281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1055/2081] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:20 2022] DELETE /identity/v3/users/29ed1efc55254ca2bd1936153bba673f => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:42:20.772657 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6088e63-94d7-4693-a57f-dc082d58074a None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.772971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6088e63-94d7-4693-a57f-dc082d58074a None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.775141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6088e63-94d7-4693-a57f-dc082d58074a None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.775660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1027/2082] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:42:20.781399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.783061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.784209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.821611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:20.822948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:20.879716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1056/2083] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:42:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:20.892348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6088e63-94d7-4693-a57f-dc082d58074a None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:20.894307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d6088e63-94d7-4693-a57f-dc082d58074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:20.912476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6088e63-94d7-4693-a57f-dc082d58074a None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:20.916354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bf165-1a70-4d9d-9f79-6202b4d25f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:20.917061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bf165-1a70-4d9d-9f79-6202b4d25f69 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:20.922033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bf165-1a70-4d9d-9f79-6202b4d25f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:20.922033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bf165-1a70-4d9d-9f79-6202b4d25f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:20.926366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bf165-1a70-4d9d-9f79-6202b4d25f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:20.928058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bf165-1a70-4d9d-9f79-6202b4d25f69 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:20.939489 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1028/2084] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:20.983768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:20.985586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:21.044530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee850b-3b2e-487c-a4ec-871075d7ad73 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.045196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee850b-3b2e-487c-a4ec-871075d7ad73 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.047877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee850b-3b2e-487c-a4ec-871075d7ad73 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.047877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1029/2085] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 14:42:21.053710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.054373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.054855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.074600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036457-6678-4ebf-a5c2-c46691c2ea3c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:21.076896 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:21.078144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:21.080126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:21.080757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.081080 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.081359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] PATH_INFO: `/v3/projects/57ea3f28b9714305af35c7e5badf8888` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.082474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ea3f28b9714305af35c7e5badf8888)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:21.084403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:21.154027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1030/2086] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:42:21.165679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:21.175767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5a689-538e-4b51-bdc8-c6d36b4080a2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:21.179482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31779b-883c-43b6-a568-7f496047c646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:21.180095 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31779b-883c-43b6-a568-7f496047c646 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.180374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31779b-883c-43b6-a568-7f496047c646 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.180644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31779b-883c-43b6-a568-7f496047c646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.196956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31779b-883c-43b6-a568-7f496047c646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:21.196956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31779b-883c-43b6-a568-7f496047c646 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:21.204324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1031/2087] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:42:21.224962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:21.233743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ea3f28b9714305af35c7e5badf8888', 'request_id': 'req-58a96369-e962-48a4-86ac-32071d1fc352'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:21.243755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ea3f28b9714305af35c7e5badf8888', 'request_id': 'req-58a96369-e962-48a4-86ac-32071d1fc352'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:21.369566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ea3f28b9714305af35c7e5badf8888', 'request_id': 'req-58a96369-e962-48a4-86ac-32071d1fc352'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:21.393051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] The token cache is being invalidate because project 57ea3f28b9714305af35c7e5badf8888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:21.393769 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:21.394090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58a96369-e962-48a4-86ac-32071d1fc352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:21.396071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1057/2088] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:20 2022] DELETE /identity/v3/projects/57ea3f28b9714305af35c7e5badf8888 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:21.401217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.401905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.402420 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.432833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:21.436118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:21.504072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:21.556138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1032/2089] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:21.568273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.568670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.568950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.611647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:21.611998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:21.620961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1058/2090] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:21.625975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:21.647515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371cbe7-14e3-44cd-b075-ce2646987350 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:21.671167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:21.675790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.676035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.676312 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.676988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:42:21.685725 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:21.687393 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:21.701456 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1033/2091] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:42:21.706233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:21.717941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334c657-f6b2-4c57-9e98-b4a0eb470eda None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:21.723843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:21.724456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.724730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.725018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.725662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:21.727593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:21.815107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:21.815286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1059/2092] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:21 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:21.825215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a78f5-9082-4534-9793-2c0858d06696 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:21.901422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:21.902031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:21.902310 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:21.903381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:21.904018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:21.906074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:21.988698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1034/2093] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:21 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:21.991977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:22.004657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f10f5-0c15-4165-9ab9-6d5047343250 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:22.009501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:22.010728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:22.011004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:22.011273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:22.011912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:22.013737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:22.024774 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1060/2094] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:42:22.029260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:22.041308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ca02-49fa-4b71-89b3-3940434da3bc admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:22.046392 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bfe69-e61f-4484-8ff0-1631ea8951b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:22.049077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bfe69-e61f-4484-8ff0-1631ea8951b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:22.049077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bfe69-e61f-4484-8ff0-1631ea8951b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:22.049642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bfe69-e61f-4484-8ff0-1631ea8951b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:22.050122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bfe69-e61f-4484-8ff0-1631ea8951b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:22.051770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bfe69-e61f-4484-8ff0-1631ea8951b7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:22.085449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1035/2095] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:22.090079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:22.103887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcff3e-b5d1-4ae2-8bb5-e9402ae819f0 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:22.109687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:22.112286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:22.112761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:22.113280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] PATH_INFO: `/v3/projects/0bf85a3ad76149e9a71142eda94141f6/users/cd5d95880554496d8d4e55a29cb18ba1/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:22.132902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf85a3ad76149e9a71142eda94141f6, user_id=cd5d95880554496d8d4e55a29cb18ba1, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:22.135833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:22.220687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1061/2096] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:22 2022] PUT /identity/v3/projects/0bf85a3ad76149e9a71142eda94141f6/users/cd5d95880554496d8d4e55a29cb18ba1/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:42:22.232008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70157-f6fe-45ea-a61e-5d2e4f1f4692 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:22.232829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70157-f6fe-45ea-a61e-5d2e4f1f4692 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:22.232829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70157-f6fe-45ea-a61e-5d2e4f1f4692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:22.267169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-08e70157-f6fe-45ea-a61e-5d2e4f1f4692 None None] MFA Rules not processed for user `cd5d95880554496d8d4e55a29cb18ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:22.268543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-08e70157-f6fe-45ea-a61e-5d2e4f1f4692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:22.361496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1036/2097] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:22.372761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:22.374800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:22.478317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49724-252c-47bb-a2cb-edef1ebe0d62 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:22.506674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:22.507312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:22.507595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:22.507864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:22.519766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:22.524186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:22.554136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1062/2098] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:30.902312 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:30.902813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:30.902906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:30.926353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:30.927454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:30.965137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:30.998783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1037/2099] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:31.024515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:31.026180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:31.108207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5db00c-70f4-427b-a951-b2c381d63bf8 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:31.114288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2131f8-af70-471f-96a2-130bb7e48b09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:31.114914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2131f8-af70-471f-96a2-130bb7e48b09 service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:31.115214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2131f8-af70-471f-96a2-130bb7e48b09 service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:31.115472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2131f8-af70-471f-96a2-130bb7e48b09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:31.125164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2131f8-af70-471f-96a2-130bb7e48b09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:31.127613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2131f8-af70-471f-96a2-130bb7e48b09 service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:31.131138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:31.136630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1063/2100] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:42:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:42:32.877293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:32.991999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d5a27-d43a-4f22-a3cf-ccb64b077997 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:32.997061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:32.997709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:32.997986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:32.998260 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] PATH_INFO: `/v3/users/cd5d95880554496d8d4e55a29cb18ba1` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:32.999484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5d95880554496d8d4e55a29cb18ba1)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:33.002867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:33.191807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5d95880554496d8d4e55a29cb18ba1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:33.206880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5d95880554496d8d4e55a29cb18ba1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:33.285800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5d95880554496d8d4e55a29cb18ba1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:33.286789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5d95880554496d8d4e55a29cb18ba1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:33.287724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5d95880554496d8d4e55a29cb18ba1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:33.306446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1038/2101] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:32 2022] DELETE /identity/v3/users/cd5d95880554496d8d4e55a29cb18ba1 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:33.308761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:33.309291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:33.309490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:33.331522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:33.332759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:33.398313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1064/2102] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:33.408499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:33.409973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:33.473349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1ecc3-d153-4827-b66a-8ad2b8eb58d9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:33.477952 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947881f-81c4-45f5-9a19-06c8e162e9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:33.479069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947881f-81c4-45f5-9a19-06c8e162e9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:33.479574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947881f-81c4-45f5-9a19-06c8e162e9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:33.480053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947881f-81c4-45f5-9a19-06c8e162e9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:33.493138 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947881f-81c4-45f5-9a19-06c8e162e9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:33.494724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947881f-81c4-45f5-9a19-06c8e162e9e3 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:33.501993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:33.508570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1039/2103] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:34.537760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:34.539946 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:34.558891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0326e6-d6fb-4cdf-9fb5-1ae29afe4dc4 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:34.565760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:34.568216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:34.568216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:34.568216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] PATH_INFO: `/v3/projects/0bf85a3ad76149e9a71142eda94141f6` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:34.568703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf85a3ad76149e9a71142eda94141f6)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:34.570670 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:34.697127 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf85a3ad76149e9a71142eda94141f6', 'request_id': 'req-43a3d3b4-e122-49db-bbad-896c35770303'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:34.705260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf85a3ad76149e9a71142eda94141f6', 'request_id': 'req-43a3d3b4-e122-49db-bbad-896c35770303'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:34.775559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf85a3ad76149e9a71142eda94141f6', 'request_id': 'req-43a3d3b4-e122-49db-bbad-896c35770303'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:34.790833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] The token cache is being invalidate because project 0bf85a3ad76149e9a71142eda94141f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:34.793732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:34.793732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43a3d3b4-e122-49db-bbad-896c35770303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:34.793732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1065/2104] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:34 2022] DELETE /identity/v3/projects/0bf85a3ad76149e9a71142eda94141f6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:34.799675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:34.800215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:34.800215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:34.822720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:34.823881 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:34.894437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1040/2105] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:34.902348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703479-2656-4498-a710-ca0729a3f205 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:34.902714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703479-2656-4498-a710-ca0729a3f205 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:34.903088 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703479-2656-4498-a710-ca0729a3f205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:34.926563 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-be703479-2656-4498-a710-ca0729a3f205 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:34.927886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-be703479-2656-4498-a710-ca0729a3f205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:34.936057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1066/2106] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:42:34.941574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:34.952259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25da9c7-492c-491d-9ad8-8078348fd485 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:34.958441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:34.959146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:34.959497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:34.959834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:34.960485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:42:34.960783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:34.962443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:34.976346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1041/2107] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:42:34.980595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be703479-2656-4498-a710-ca0729a3f205 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:34.996039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be703479-2656-4498-a710-ca0729a3f205 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.001099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.001792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.002078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.002348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.002998 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.004915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:35.110671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1067/2108] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:34 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:35.114899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:35.131830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c275e2-4d98-479a-9d09-811b5e9de7c8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.227901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.228712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.228993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.229418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.230157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.233212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:35.324133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1042/2109] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:35 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:42:35.325371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:35.347588 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa617907-6cbe-4210-9034-69c934c59dde admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.352508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.353159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.353436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.353706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.354348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.356314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:35.368397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1068/2110] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:35.372929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:35.384295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b868e1-888b-4769-bd2b-644dcc62059f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.388942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f9e8f-4771-4908-966b-3575b1731e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.389593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f9e8f-4771-4908-966b-3575b1731e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.389869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f9e8f-4771-4908-966b-3575b1731e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.390133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f9e8f-4771-4908-966b-3575b1731e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.390764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f9e8f-4771-4908-966b-3575b1731e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.393909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f9e8f-4771-4908-966b-3575b1731e6c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:35.407525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1043/2111] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:42:35.412022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:35.430723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e2012-5e40-4a0b-a892-6a9d34a86eea admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.435805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.436657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.437150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.437530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] PATH_INFO: `/v3/projects/3845c486be3541e59f186762c5fa2e37/users/d84e3893d5354cda94b26cfaebede211/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.459235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3845c486be3541e59f186762c5fa2e37, user_id=d84e3893d5354cda94b26cfaebede211, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.462133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:35.541949 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1069/2112] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:35 2022] PUT /identity/v3/projects/3845c486be3541e59f186762c5fa2e37/users/d84e3893d5354cda94b26cfaebede211/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:35.554988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.560750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.560750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.588082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] MFA Rules not processed for user `d84e3893d5354cda94b26cfaebede211`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:35.589433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:35.665617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:35.676811 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1044/2113] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:35.682276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:35.703671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:35.752547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab367-6acc-4e66-8d2d-59726d86cbb1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.758154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.758903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.759202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.759645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.760362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.762836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:35.881748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1070/2114] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:35 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:35.885803 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:35.898835 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087d8a2-5732-402a-8789-ca9c930c2d64 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:35.972014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:35.972634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:35.972936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:35.973219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:35.973862 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:35.975792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:36.072120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1045/2115] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:35 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:36.073343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:36.084965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0889a0-adbc-441a-a4cb-eb4d460cb87d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:36.089761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:36.090357 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:36.090632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:36.090901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:36.091555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:36.093262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:36.110041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1071/2116] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:42:36.111814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:36.123102 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c1b3-a620-4e26-bea3-7fb91f6d4ae9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:36.132467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea4479-979b-4749-9dd4-cc25cf903055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:36.133052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4479-979b-4749-9dd4-cc25cf903055 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:36.133384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4479-979b-4749-9dd4-cc25cf903055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:36.134978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4479-979b-4749-9dd4-cc25cf903055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:36.135655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea4479-979b-4749-9dd4-cc25cf903055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:36.137366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea4479-979b-4749-9dd4-cc25cf903055 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:36.165884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1046/2117] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:36.169924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:36.183453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406fca-d84c-45d4-b247-96b916b68156 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:36.190237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:36.195038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:36.196008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:36.196685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] PATH_INFO: `/v3/projects/aae1f360509647c0a12257bdc768ef93/users/7f4822804e9945f2b848faca04af0a03/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:36.219183 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae1f360509647c0a12257bdc768ef93, user_id=7f4822804e9945f2b848faca04af0a03, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:36.225293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:36.323436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1072/2118] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:36 2022] PUT /identity/v3/projects/aae1f360509647c0a12257bdc768ef93/users/7f4822804e9945f2b848faca04af0a03/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:36.347284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:36.347284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:36.347284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:36.406810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] MFA Rules not processed for user `7f4822804e9945f2b848faca04af0a03`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:36.408735 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:36.546067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1047/2119] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:36.550770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:36.684405 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf906-a5ab-4d43-9418-3415efffe359 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:36.691591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:36.692425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:36.692802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:36.693202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:36.694106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:36.696827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:36.829652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1073/2120] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:36 2022] POST /identity/v3/projects => generated 375 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:36.833779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:36.848539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373101e6-b7fb-42cb-9256-c7c1080ed630 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:36.935249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:36.936089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:36.936089 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:36.936411 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:36.937137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:36.939438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:36.950509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:37.056654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1048/2121] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:36 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:37.060433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:37.072398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eab49f-f780-4905-af90-e242c9d5c60a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:37.077172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:37.077794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:37.078074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:37.078349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:37.079020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:37.083254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:37.093759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1074/2122] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:37.098814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:37.112090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f03cc8-45e4-424b-99af-f177c02f0996 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:37.120390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:37.121154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:37.121443 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:37.121752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] PATH_INFO: `/v3/projects/65ff6b56d28a4fe38496bda7c89a29bc/users/a16f94b5e72e411f985f8960a2f69276/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:37.141330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ff6b56d28a4fe38496bda7c89a29bc, user_id=a16f94b5e72e411f985f8960a2f69276, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:37.145480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:37.210924 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1049/2123] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:37 2022] PUT /identity/v3/projects/65ff6b56d28a4fe38496bda7c89a29bc/users/a16f94b5e72e411f985f8960a2f69276/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:42:37.215151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:37.226063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235eda02-82f2-4d16-b31d-f5422b88c11e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:37.289678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:37.307188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42388f-ef7e-46f9-a7b5-6a5d1e4edb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:37.307797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42388f-ef7e-46f9-a7b5-6a5d1e4edb51 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:37.308073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42388f-ef7e-46f9-a7b5-6a5d1e4edb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:37.308341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42388f-ef7e-46f9-a7b5-6a5d1e4edb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:37.308974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42388f-ef7e-46f9-a7b5-6a5d1e4edb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:37.310729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42388f-ef7e-46f9-a7b5-6a5d1e4edb51 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:37.327467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1075/2124] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:42:37.328760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:37.340892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2baa7-d42c-47b0-b162-666145425f24 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:37.346015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:37.346684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:37.346976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:37.347229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] PATH_INFO: `/v3/projects/65ff6b56d28a4fe38496bda7c89a29bc/users/a16f94b5e72e411f985f8960a2f69276/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:37.349259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ff6b56d28a4fe38496bda7c89a29bc, user_id=a16f94b5e72e411f985f8960a2f69276, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:37.352410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:37.381050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1050/2125] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:37 2022] PUT /identity/v3/projects/65ff6b56d28a4fe38496bda7c89a29bc/users/a16f94b5e72e411f985f8960a2f69276/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:37.391491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:37.391798 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:37.392071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:37.422136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] MFA Rules not processed for user `a16f94b5e72e411f985f8960a2f69276`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:37.424185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:37.506365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1076/2126] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:42:37.513741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:37.515591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:37.599734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930f7b-2668-46b5-bf96-874a1911d1d3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:37.604124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:37.604875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:37.605188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:37.605525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:37.680157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:37.681823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:37.693256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1051/2127] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:39.939228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:39.950057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53d4f4-2709-416e-8caa-6be0fd29bad3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:39.954016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0e33-4f74-40b7-9e1b-9786174eafbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:39.955017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0e33-4f74-40b7-9e1b-9786174eafbb service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:39.955406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0e33-4f74-40b7-9e1b-9786174eafbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:39.955692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0e33-4f74-40b7-9e1b-9786174eafbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:40.037708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a0e33-4f74-40b7-9e1b-9786174eafbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:40.039645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a0e33-4f74-40b7-9e1b-9786174eafbb service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:40.057598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1077/2128] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:41.524061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:41.534085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e060013-5b8d-42c7-b2fa-a13f346473fe service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:41.538329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a21273-8227-42d1-9703-2585ad885c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:41.539068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a21273-8227-42d1-9703-2585ad885c26 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:41.539448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a21273-8227-42d1-9703-2585ad885c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:41.539821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a21273-8227-42d1-9703-2585ad885c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:41.551074 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a21273-8227-42d1-9703-2585ad885c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:41.552800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a21273-8227-42d1-9703-2585ad885c26 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:41.564834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1052/2129] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:45.596851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434b2dc-53f3-4b4d-a04b-883842c6c0ff None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:45.597421 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434b2dc-53f3-4b4d-a04b-883842c6c0ff None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:45.597532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434b2dc-53f3-4b4d-a04b-883842c6c0ff None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:45.598839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1078/2130] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 14:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:42:45.604029 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:45.604542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:45.604542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:45.625742 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:45.628172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:45.687946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:45.713192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1053/2131] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:45.719023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434b2dc-53f3-4b4d-a04b-883842c6c0ff None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:45.735885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434b2dc-53f3-4b4d-a04b-883842c6c0ff None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:45.742971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:45.743194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:45.743477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:45.743807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:45.746415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:45.759853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:45.767190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:45.777267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1079/2132] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:46.181594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:46.182986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:46.204340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:46.258229 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:46.262099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fbdd7-e2d8-4dfa-8ed8-6f340401cc7f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:46.266012 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:46.269251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:46.269251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:46.269251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:46.269696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:46.276146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59437a0-7e1d-4c17-8666-13b7447d226b service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:46.281306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:46.282010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:46.282327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:46.282548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] PATH_INFO: `/v3/users/d84e3893d5354cda94b26cfaebede211` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:46.283754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84e3893d5354cda94b26cfaebede211)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:46.285787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:46.349537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:46.353659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35e74-3171-49bd-9e95-3750364e2d43 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:46.362066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1054/2133] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:46.464419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84e3893d5354cda94b26cfaebede211'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:46.536979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84e3893d5354cda94b26cfaebede211'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:46.586008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84e3893d5354cda94b26cfaebede211'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:46.587913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84e3893d5354cda94b26cfaebede211'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:46.588689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84e3893d5354cda94b26cfaebede211'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:46.626943 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1080/2134] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:46 2022] DELETE /identity/v3/users/d84e3893d5354cda94b26cfaebede211 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:42:46.631633 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:46.632108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:46.632249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:46.654177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:46.655893 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:46.747579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1055/2135] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:46.758768 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:46.761083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:46.825910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a47b1-db51-4ab9-8f0d-d0f904cfec74 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:46.830358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:46.831035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:46.831241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:46.831575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:46.841273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:46.842698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:46.853320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1081/2136] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:42:47.292223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:47.293992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:47.314060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72688c87-95b1-4220-84a7-292f4932e8a6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:47.329653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:47.329653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:47.329653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:47.329653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] PATH_INFO: `/v3/projects/3845c486be3541e59f186762c5fa2e37` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:47.331130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=3845c486be3541e59f186762c5fa2e37)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:47.333140 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:47.473113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:47.482852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3845c486be3541e59f186762c5fa2e37', 'request_id': 'req-9889a3c5-49b2-4938-b66c-99736e375cce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:47.490845 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3845c486be3541e59f186762c5fa2e37', 'request_id': 'req-9889a3c5-49b2-4938-b66c-99736e375cce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:47.497718 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3845c486be3541e59f186762c5fa2e37', 'request_id': 'req-9889a3c5-49b2-4938-b66c-99736e375cce'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:47.633900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] The token cache is being invalidate because project 3845c486be3541e59f186762c5fa2e37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:47.634546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:47.634851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:47.638478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1056/2137] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:47 2022] DELETE /identity/v3/projects/3845c486be3541e59f186762c5fa2e37 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:47.642181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:47.652551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b3c9c-44f8-4f43-8dec-c9961954bb6c service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:47.730692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:47.731382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:47.731736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:47.732017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] PATH_INFO: `/v3/users/7f4822804e9945f2b848faca04af0a03` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:47.734147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4822804e9945f2b848faca04af0a03)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:47.738792 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:47.768493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:48.009767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4822804e9945f2b848faca04af0a03'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.095843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4822804e9945f2b848faca04af0a03'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.122480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4822804e9945f2b848faca04af0a03'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.122480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4822804e9945f2b848faca04af0a03'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.122480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4822804e9945f2b848faca04af0a03'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.132446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1082/2138] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:47 2022] DELETE /identity/v3/users/7f4822804e9945f2b848faca04af0a03 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:48.587916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:48.600350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:48.616661 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889a3c5-49b2-4938-b66c-99736e375cce admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:48.718015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:48.719541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:48.720608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:48.721024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] PATH_INFO: `/v3/projects/aae1f360509647c0a12257bdc768ef93` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:48.722705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae1f360509647c0a12257bdc768ef93)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:48.725026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:48.845960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae1f360509647c0a12257bdc768ef93', 'request_id': 'req-42feb732-38d2-47b7-bd56-39e74a5a4f77'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.854445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae1f360509647c0a12257bdc768ef93', 'request_id': 'req-42feb732-38d2-47b7-bd56-39e74a5a4f77'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.865047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae1f360509647c0a12257bdc768ef93', 'request_id': 'req-42feb732-38d2-47b7-bd56-39e74a5a4f77'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.981629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] The token cache is being invalidate because project aae1f360509647c0a12257bdc768ef93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:48.982022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.982746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:48.985745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1057/2139] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:48 2022] DELETE /identity/v3/projects/aae1f360509647c0a12257bdc768ef93 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:48.991567 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:48.994085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:49.007374 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5164-b6f4-4b54-95cf-051dee3537a9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:49.075644 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:49.076747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:49.077049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:49.077527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] PATH_INFO: `/v3/users/a16f94b5e72e411f985f8960a2f69276` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:49.080882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16f94b5e72e411f985f8960a2f69276)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:49.083258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:49.278661 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16f94b5e72e411f985f8960a2f69276'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:49.368754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16f94b5e72e411f985f8960a2f69276'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:49.395679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16f94b5e72e411f985f8960a2f69276'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:49.396340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16f94b5e72e411f985f8960a2f69276'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:49.397169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fe6a3f8a-3342-4993-9a32-f5a104cd8160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16f94b5e72e411f985f8960a2f69276'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:49.412300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1083/2140] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:48 2022] DELETE /identity/v3/users/a16f94b5e72e411f985f8960a2f69276 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:42:49.867873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:49.870266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:49.893712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feb732-38d2-47b7-bd56-39e74a5a4f77 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:49.966957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:49.976530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:49.977345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:49.977704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:49.978050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] PATH_INFO: `/v3/projects/65ff6b56d28a4fe38496bda7c89a29bc` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:49.979410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ff6b56d28a4fe38496bda7c89a29bc)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:49.989524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.106355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ff6b56d28a4fe38496bda7c89a29bc', 'request_id': 'req-243d2493-a1a3-4029-a124-aa4e04f88e5d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:50.115616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ff6b56d28a4fe38496bda7c89a29bc', 'request_id': 'req-243d2493-a1a3-4029-a124-aa4e04f88e5d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:50.123167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ff6b56d28a4fe38496bda7c89a29bc', 'request_id': 'req-243d2493-a1a3-4029-a124-aa4e04f88e5d'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:50.198717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] The token cache is being invalidate because project 65ff6b56d28a4fe38496bda7c89a29bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:42:50.199141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:50.199689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-243d2493-a1a3-4029-a124-aa4e04f88e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:50.201551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1058/2141] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:42:49 2022] DELETE /identity/v3/projects/65ff6b56d28a4fe38496bda7c89a29bc => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:50.206641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.207185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.207185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.227123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:50.228319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:50.232490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:50.289810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1084/2142] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:50.359404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.359691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.360726 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.381262 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:50.382519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:50.390536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1059/2143] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:50.395162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:50.405289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae70a9-487d-4a8e-b62e-a402c77b8bed None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:50.411933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:50.412537 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.412816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.413125 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.413786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:42:50.414104 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:50.415681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.432260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1085/2144] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:42:50.433437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:50.443413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278d741-0eb1-48eb-9b21-46a1a77558bb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:50.448211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:50.448823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.449137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.449416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.450466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:50.452108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.565761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1060/2145] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:42:50 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:50.569992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:50.580072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921db5a-2d29-4900-b8d0-c84d355e9f03 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:50.637025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:50.637875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.638339 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.639237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.639882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:50.642073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.759495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1086/2146] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:42:50 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:42:50.763542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:50.776627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11b8d8-bda4-4046-b4a0-8d3cd748e9c8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:50.781495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:50.782751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.782972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.783384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.784028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:50.785671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.799118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1061/2147] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:42:50.800264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:50.810305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8308ee6-e22b-4f66-8378-70d7622baa9c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:50.814926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24430464-015b-4458-bebc-c31c6d0de048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:50.815598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24430464-015b-4458-bebc-c31c6d0de048 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.815785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24430464-015b-4458-bebc-c31c6d0de048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.816047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24430464-015b-4458-bebc-c31c6d0de048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.816683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24430464-015b-4458-bebc-c31c6d0de048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:50.818314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24430464-015b-4458-bebc-c31c6d0de048 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.831731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1087/2148] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:42:50.835922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:50.846866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca931a-eb73-4dd6-849d-81625caa4ef5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:50.851805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:50.852476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.852750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.853043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] PATH_INFO: `/v3/projects/9c12389bda984f2bacc5e7211c4704c0/users/c58f142a8dba44a98210d2586196ecb5/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:50.869523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c12389bda984f2bacc5e7211c4704c0, user_id=c58f142a8dba44a98210d2586196ecb5, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:50.872181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:50.968713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1062/2149] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:42:50 2022] PUT /identity/v3/projects/9c12389bda984f2bacc5e7211c4704c0/users/c58f142a8dba44a98210d2586196ecb5/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:42:50.984303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883740eb-ffbf-4a91-8151-e5296134222b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:50.984303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883740eb-ffbf-4a91-8151-e5296134222b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:50.984303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883740eb-ffbf-4a91-8151-e5296134222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:51.011248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-883740eb-ffbf-4a91-8151-e5296134222b None None] MFA Rules not processed for user `c58f142a8dba44a98210d2586196ecb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:51.012473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-883740eb-ffbf-4a91-8151-e5296134222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:51.100967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1088/2150] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:51.114113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:51.115855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:51.245557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906da90-74a1-4447-a738-0a22c72a9f7f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:51.252248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:51.252883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:51.253201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:51.253700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:51.265619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:51.266020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:51.277768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1063/2151] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:42:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:42:54.366295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e78d6c-2f86-4e75-9d90-1cf7a47a5579 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:54.366833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e78d6c-2f86-4e75-9d90-1cf7a47a5579 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:54.366833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e78d6c-2f86-4e75-9d90-1cf7a47a5579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:54.375264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:54.387220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-96e78d6c-2f86-4e75-9d90-1cf7a47a5579 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:54.388426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-96e78d6c-2f86-4e75-9d90-1cf7a47a5579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:54.446680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1089/2152] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:42:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:54.459404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:54.460830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:54.492760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:42:54.532789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebd354-e099-4b40-a72c-93fc00dcc9a3 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:54.538126 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:54.538740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:54.539016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:54.539285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:54.548423 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:54.550055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:54.558666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1064/2153] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:42:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:42:55.357754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd703d-d3e4-4d00-9604-ddb03a31f03c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:55.358605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd703d-d3e4-4d00-9604-ddb03a31f03c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:55.359064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd703d-d3e4-4d00-9604-ddb03a31f03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:55.382730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-69fd703d-d3e4-4d00-9604-ddb03a31f03c None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:55.384662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-69fd703d-d3e4-4d00-9604-ddb03a31f03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:55.392246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1090/2154] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:55.409956 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:55.417386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c566-efa7-434f-8fab-79b3b8e25122 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:55.423090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:55.423717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:55.423992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:55.424263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:55.432548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:55.434163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:55.443088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1065/2155] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:42:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:42:56.009262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419b29-2927-4901-8caf-3e5c03c2f35c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:56.009547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419b29-2927-4901-8caf-3e5c03c2f35c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:56.009826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419b29-2927-4901-8caf-3e5c03c2f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:56.029805 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-b3419b29-2927-4901-8caf-3e5c03c2f35c None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:56.031048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b3419b29-2927-4901-8caf-3e5c03c2f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:56.039304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1091/2156] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:42:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:56.056270 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:56.066844 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4086c4-420a-40ce-a807-96979d9f7fa7 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:56.071941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:56.072588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:56.072872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:56.073173 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:56.082033 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:56.083618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:56.091610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1066/2157] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:42:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:42:56.680423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952789e-72dd-48fc-9e9e-f5fc26517742 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:56.680901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952789e-72dd-48fc-9e9e-f5fc26517742 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:56.680990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952789e-72dd-48fc-9e9e-f5fc26517742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:56.699883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a952789e-72dd-48fc-9e9e-f5fc26517742 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:56.701128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a952789e-72dd-48fc-9e9e-f5fc26517742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:56.708716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1092/2158] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 14:42:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:56.721701 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:56.732032 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083718b-b418-40c9-8930-6c95f8887b75 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:56.737303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:56.737885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:56.738158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:56.738424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:56.746462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:56.748030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:56.756288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1067/2159] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:42:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:42:58.476723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:58.476723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:58.476723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:58.498145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:42:58.499368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:58.507312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1093/2160] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:42:58.520497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:58.531458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bc9f0-7eb5-4329-87df-8f755e138e22 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:58.537708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd8fb3-7a77-455e-b8df-4232500854c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:58.538304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd8fb3-7a77-455e-b8df-4232500854c3 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:58.538580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd8fb3-7a77-455e-b8df-4232500854c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:58.538846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd8fb3-7a77-455e-b8df-4232500854c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:58.548557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd8fb3-7a77-455e-b8df-4232500854c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:58.550201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd8fb3-7a77-455e-b8df-4232500854c3 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:58.559427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1068/2161] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:42:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:42:59.355689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:42:59.357228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:42:59.425855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc6536-163c-4a66-8f9a-35d1531a564b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:42:59.430341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:42:59.430961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:59.431236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:59.431506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] PATH_INFO: `/v3/users/2fa6a7aba33d4abd9389ecd2e3fa0906` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:42:59.432618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa6a7aba33d4abd9389ecd2e3fa0906)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:42:59.434648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:42:59.680087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa6a7aba33d4abd9389ecd2e3fa0906'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:59.891603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa6a7aba33d4abd9389ecd2e3fa0906'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:59.922780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa6a7aba33d4abd9389ecd2e3fa0906'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:59.922780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa6a7aba33d4abd9389ecd2e3fa0906'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:59.922780 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-85bda03b-f4ab-4b26-9574-14d1892e9e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa6a7aba33d4abd9389ecd2e3fa0906'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:42:59.968649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1094/2162] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:42:59 2022] DELETE /identity/v3/users/2fa6a7aba33d4abd9389ecd2e3fa0906 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:42:59.982354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:42:59.982919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:42:59.983143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:00.033307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:00.035449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:00.035942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:00.036402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:00.037711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:00.052396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:00.100561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:00.101882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:00.252055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1069/2163] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:43:00.259955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:00.264043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:00.351655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1095/2164] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 14:43:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:43:00.369138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:00.370788 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:00.429569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2aed2-4c93-4952-bef7-cf28be11c85f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:00.450092 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:00.457655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:00.461267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:00.461703 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:00.518581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:00.518581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:00.527579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1070/2165] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:43:00.622202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60898987-5a22-4919-a5c8-20fd24a1500d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:00.636057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db631e-35dc-4f44-97c9-0ea9793e788f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:00.636696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db631e-35dc-4f44-97c9-0ea9793e788f service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:00.636971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db631e-35dc-4f44-97c9-0ea9793e788f service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:00.637372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db631e-35dc-4f44-97c9-0ea9793e788f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:00.667057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db631e-35dc-4f44-97c9-0ea9793e788f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:00.669241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db631e-35dc-4f44-97c9-0ea9793e788f service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:00.688655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1096/2166] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:43:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:43:01.148558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:01.151055 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:01.177284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f717b5-c146-4432-bfb7-9b7fd45c8baf service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:01.185594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:01.186508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:01.186945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:01.187383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] PATH_INFO: `/v3/projects/f8e090451c7e4296a0b85c7eeb9002f7` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:01.189375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e090451c7e4296a0b85c7eeb9002f7)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:01.191610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:01.195329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:01.400013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e090451c7e4296a0b85c7eeb9002f7', 'request_id': 'req-44f3ff02-8bc2-4990-9295-925033845f1b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:01.408738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e090451c7e4296a0b85c7eeb9002f7', 'request_id': 'req-44f3ff02-8bc2-4990-9295-925033845f1b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:01.431002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e090451c7e4296a0b85c7eeb9002f7', 'request_id': 'req-44f3ff02-8bc2-4990-9295-925033845f1b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:01.612770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] The token cache is being invalidate because project f8e090451c7e4296a0b85c7eeb9002f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:43:01.613220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:01.619266 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-44f3ff02-8bc2-4990-9295-925033845f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:01.620885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1071/2167] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:43:01 2022] DELETE /identity/v3/projects/f8e090451c7e4296a0b85c7eeb9002f7 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:43:01.626888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:01.627191 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:01.627346 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:01.686906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:01.690384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:01.838510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1097/2168] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:01.915018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:01.915018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:01.915018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:01.939318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:01.941801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:01.955582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1072/2169] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:43:01.959889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:01.973254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d8a73-6a24-4e6c-9f78-52e1c3e34bcc None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:02.004790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:02.005862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.006288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.006734 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.007599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:43:02.008067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:02.010037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:02.029655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1098/2170] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:43:02.035101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:02.057283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c48ea-f0b2-45cd-89bd-1e8e04620d64 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:02.082593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:02.085681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.087544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.089855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.092479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:02.097615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:02.197137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1073/2171] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:43:02 2022] POST /identity/v3/projects => generated 391 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:43:02.201345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:02.220097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb822c-f215-414c-a205-cce97556d4b1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:02.250282 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:02.296073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:02.296705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.297113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.297868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.298735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:02.300863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:02.425662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1099/2172] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:43:02 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:43:02.429708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:02.458153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1732380-eed1-4585-918e-5c7d770cfae1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:02.475195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:02.476317 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.479468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.480227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.482505 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:02.489490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:02.509832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1074/2173] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:43:02.513937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:02.537131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675e0f8-f3d0-4d19-880c-2a5a76c51ce9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:02.549994 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4702bb-6d68-4567-8574-8802986b5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:02.556081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4702bb-6d68-4567-8574-8802986b5578 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.556081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4702bb-6d68-4567-8574-8802986b5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.556081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4702bb-6d68-4567-8574-8802986b5578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.556081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4702bb-6d68-4567-8574-8802986b5578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:02.556081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4702bb-6d68-4567-8574-8802986b5578 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:02.576803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1100/2174] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:43:02.595951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:02.618629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f2838-c46e-4f15-824f-99fc61ddcc20 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:02.626105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:02.626105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.626616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.627059 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] PATH_INFO: `/v3/projects/99bff06564564ba09efad3aebe0efc27/users/040fce1acf5e440c8b0cf4296a1a3b64/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.649951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bff06564564ba09efad3aebe0efc27, user_id=040fce1acf5e440c8b0cf4296a1a3b64, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:02.654770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:02.763158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1075/2175] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:43:02 2022] PUT /identity/v3/projects/99bff06564564ba09efad3aebe0efc27/users/040fce1acf5e440c8b0cf4296a1a3b64/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:43:02.775937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:02.776912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:02.777468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:02.830863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] MFA Rules not processed for user `040fce1acf5e440c8b0cf4296a1a3b64`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:02.832433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:03.008525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1101/2176] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:03.021652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:03.024808 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:03.135257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640390c6-5969-445c-b6c1-2c29acaa2394 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:03.135257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561e941-2c32-44fb-be95-1770e6175fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:03.135257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561e941-2c32-44fb-be95-1770e6175fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:03.135257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561e941-2c32-44fb-be95-1770e6175fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:03.136122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561e941-2c32-44fb-be95-1770e6175fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:03.138990 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561e941-2c32-44fb-be95-1770e6175fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:03.140036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561e941-2c32-44fb-be95-1770e6175fa6 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:03.151472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1076/2177] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:43:13.005170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:13.146704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:14.655663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:14.655663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:14.754363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9b356-0bcc-41d4-8a2d-a26f2ea6e027 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:14.759039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:14.764606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:14.764606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:14.764606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] PATH_INFO: `/v3/users/c58f142a8dba44a98210d2586196ecb5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:14.764606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58f142a8dba44a98210d2586196ecb5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:14.764606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:15.007856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58f142a8dba44a98210d2586196ecb5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:15.078892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58f142a8dba44a98210d2586196ecb5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:15.102696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58f142a8dba44a98210d2586196ecb5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:15.103806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58f142a8dba44a98210d2586196ecb5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:15.104823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58f142a8dba44a98210d2586196ecb5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:15.126702 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1102/2178] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:43:14 2022] DELETE /identity/v3/users/c58f142a8dba44a98210d2586196ecb5 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:43:15.128132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:15.128780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:15.128780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:15.150816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:15.152081 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:15.229655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1077/2179] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:43:15.240271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:15.242352 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:15.320910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519db0d-83a3-438d-b089-1cf4fca1e7e5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:15.325142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad541833-ef6e-4add-a3ac-9e4e51614faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:15.325800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541833-ef6e-4add-a3ac-9e4e51614faf service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:15.326106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541833-ef6e-4add-a3ac-9e4e51614faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:15.326381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541833-ef6e-4add-a3ac-9e4e51614faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:15.338513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad541833-ef6e-4add-a3ac-9e4e51614faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:15.339950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad541833-ef6e-4add-a3ac-9e4e51614faf service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:15.351222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1103/2180] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:43:15.904813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:15.906406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:15.932014 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3bbcf-d3cb-4e1f-b18a-1e604e5b8e75 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:15.938974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:15.939640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:15.940069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:15.940351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] PATH_INFO: `/v3/projects/9c12389bda984f2bacc5e7211c4704c0` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:15.941573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c12389bda984f2bacc5e7211c4704c0)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:15.943538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:16.106780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c12389bda984f2bacc5e7211c4704c0', 'request_id': 'req-3c53f41c-6875-4371-bc25-b0d22609e623'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:16.113682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c12389bda984f2bacc5e7211c4704c0', 'request_id': 'req-3c53f41c-6875-4371-bc25-b0d22609e623'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:16.119391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c12389bda984f2bacc5e7211c4704c0', 'request_id': 'req-3c53f41c-6875-4371-bc25-b0d22609e623'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:16.204958 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] The token cache is being invalidate because project 9c12389bda984f2bacc5e7211c4704c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:43:16.205991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:16.206639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:16.208496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1078/2181] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:43:15 2022] DELETE /identity/v3/projects/9c12389bda984f2bacc5e7211c4704c0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:43:19.394280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:19.398896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:19.398896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:19.421085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:19.423202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:19.445172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:19.503192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1104/2182] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:19.519744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:19.522690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:19.613593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53f41c-6875-4371-bc25-b0d22609e623 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:19.618992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:19.619966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:19.620366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:19.620714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:19.634619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:19.636067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:19.649078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1079/2183] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:43:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 14:43:23.011616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:23.013245 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:23.080527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52e29d-38f2-4bcd-9ff7-ed794780ca61 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:23.085067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:23.085700 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:23.085977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:23.086253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] PATH_INFO: `/v3/users/c51fbcb1f2714d2c825266ec673348a0` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:23.087378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51fbcb1f2714d2c825266ec673348a0)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:23.089344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:23.247485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51fbcb1f2714d2c825266ec673348a0'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.302262 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51fbcb1f2714d2c825266ec673348a0'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.317947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51fbcb1f2714d2c825266ec673348a0'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.318668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51fbcb1f2714d2c825266ec673348a0'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.319260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8ed00001-ecec-4d32-a5e1-cfc53b587413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51fbcb1f2714d2c825266ec673348a0'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.332616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1105/2184] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:43:23 2022] DELETE /identity/v3/users/c51fbcb1f2714d2c825266ec673348a0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:43:23.673110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:23.685441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a2386-0141-4231-90d0-d2b3a21320f4 service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:23.704691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:23.759510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:23.760137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:23.760424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:23.760720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] PATH_INFO: `/v3/projects/47b8cf11e60741e29dfec5f97c462eff` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:23.761883 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b8cf11e60741e29dfec5f97c462eff)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:23.764128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:23.868416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b8cf11e60741e29dfec5f97c462eff', 'request_id': 'req-6152365a-7659-4490-bbf1-f50cd843fcb9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.874814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b8cf11e60741e29dfec5f97c462eff', 'request_id': 'req-6152365a-7659-4490-bbf1-f50cd843fcb9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.881282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b8cf11e60741e29dfec5f97c462eff', 'request_id': 'req-6152365a-7659-4490-bbf1-f50cd843fcb9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.947030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] The token cache is being invalidate because project 47b8cf11e60741e29dfec5f97c462eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:43:23.947324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.948698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-6152365a-7659-4490-bbf1-f50cd843fcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:23.949495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1080/2185] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:43:23 2022] DELETE /identity/v3/projects/47b8cf11e60741e29dfec5f97c462eff => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:43:23.955573 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:23.955863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:23.956138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:23.976507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:23.977807 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:24.022468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:24.039580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1106/2186] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:24.048477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.048767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.049076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.070543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:24.071741 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:24.080113 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1081/2187] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:24.084575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.095098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dc827-98d8-4a13-9127-56ef9312dfa3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.099935 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.102336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.102336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.102336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.102336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:43:24.102336 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.103657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.116530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1107/2188] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:43:24.120912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.130500 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d51505-cf37-4296-8ece-8f624ff30b41 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.135210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.135829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.139549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.139549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.139549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.139549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.282199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1082/2189] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:43:24 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:43:24.286412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.296595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b3bd-294f-4834-8c8c-c15eff8dced2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.363631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.364285 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.364598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.364938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.365657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.367785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.461201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1108/2190] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:43:24 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:43:24.462309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.476499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b4fb6-39d3-47ce-9ce5-ae311f834f5d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.482499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.483107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.493085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.493085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.493085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.493085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.498424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1083/2191] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:43:24.502126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.514547 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e03232-9024-48ad-9796-a23d1e3bc6e3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.521287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81843a-d2b2-4d5a-a5ea-0bd592d44f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.522263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81843a-d2b2-4d5a-a5ea-0bd592d44f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.522645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81843a-d2b2-4d5a-a5ea-0bd592d44f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.522996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81843a-d2b2-4d5a-a5ea-0bd592d44f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.523714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81843a-d2b2-4d5a-a5ea-0bd592d44f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.525543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81843a-d2b2-4d5a-a5ea-0bd592d44f36 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.541220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1109/2192] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:43:24.542211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.554102 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1fea4-e600-4b25-a0fb-bc384c226275 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.559870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.560553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.560832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.561142 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] PATH_INFO: `/v3/projects/266a4c8292344dc7989c968a30428354/users/78aaa746883547099643e5eed3768343/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.579858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266a4c8292344dc7989c968a30428354, user_id=78aaa746883547099643e5eed3768343, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.582486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.650445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1084/2193] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:43:24 2022] PUT /identity/v3/projects/266a4c8292344dc7989c968a30428354/users/78aaa746883547099643e5eed3768343/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:43:24.662224 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.662534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.662784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.704204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] MFA Rules not processed for user `78aaa746883547099643e5eed3768343`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:24.705663 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:24.772179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1110/2194] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:24.784745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.787094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:24.790261 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:24.791674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:24.832629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:24.841864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:24.846848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20cd2d-3b67-4e8d-9e85-9d822302e579 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.850444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5966514-9335-4a3b-8f8a-f3dd7205105c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.851048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966514-9335-4a3b-8f8a-f3dd7205105c service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.851323 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966514-9335-4a3b-8f8a-f3dd7205105c service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.851598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966514-9335-4a3b-8f8a-f3dd7205105c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.861038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966514-9335-4a3b-8f8a-f3dd7205105c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.862645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314b21a-9466-4f09-83a7-f5876173616e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:24.862836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966514-9335-4a3b-8f8a-f3dd7205105c service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.867215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:24.867862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:24.868153 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:24.871358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] PATH_INFO: `/v3/users/040fce1acf5e440c8b0cf4296a1a3b64` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:24.871358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] RBAC: Authorizing `identity:delete_user(user_id=040fce1acf5e440c8b0cf4296a1a3b64)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:24.871689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:24.872764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1085/2195] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:43:25.052385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '040fce1acf5e440c8b0cf4296a1a3b64'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.139847 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '040fce1acf5e440c8b0cf4296a1a3b64'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.153236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '040fce1acf5e440c8b0cf4296a1a3b64'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.153907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '040fce1acf5e440c8b0cf4296a1a3b64'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.154522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '040fce1acf5e440c8b0cf4296a1a3b64'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.167614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1111/2196] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:43:24 2022] DELETE /identity/v3/users/040fce1acf5e440c8b0cf4296a1a3b64 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:43:25.171840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:25.172263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:25.172413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:25.192285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:25.193693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:25.255991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1086/2197] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:25.265896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:25.267306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:25.324996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880984-d444-4822-9640-8b279da9198b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:25.328679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2ebdb-35b7-43f2-9383-4764f9e8b667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:25.329308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2ebdb-35b7-43f2-9383-4764f9e8b667 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:25.329787 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2ebdb-35b7-43f2-9383-4764f9e8b667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:25.329920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2ebdb-35b7-43f2-9383-4764f9e8b667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:25.338852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2ebdb-35b7-43f2-9383-4764f9e8b667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:25.340302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2ebdb-35b7-43f2-9383-4764f9e8b667 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:25.349845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1112/2198] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:43:25.805309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:43:25.806797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:25.838805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b88348-7f1e-4179-9f9d-108c50630f4b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:43:25.846199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:43:25.846917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:25.847480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:25.847755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] PATH_INFO: `/v3/projects/99bff06564564ba09efad3aebe0efc27` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:25.850009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bff06564564ba09efad3aebe0efc27)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:43:25.852576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:43:25.983275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bff06564564ba09efad3aebe0efc27', 'request_id': 'req-194a906b-b0d5-431c-9604-f26413d1bfeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.990465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bff06564564ba09efad3aebe0efc27', 'request_id': 'req-194a906b-b0d5-431c-9604-f26413d1bfeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:25.996639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bff06564564ba09efad3aebe0efc27', 'request_id': 'req-194a906b-b0d5-431c-9604-f26413d1bfeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:26.076236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] The token cache is being invalidate because project 99bff06564564ba09efad3aebe0efc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:43:26.076523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:26.077187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-194a906b-b0d5-431c-9604-f26413d1bfeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:43:26.079063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1087/2199] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:43:25 2022] DELETE /identity/v3/projects/99bff06564564ba09efad3aebe0efc27 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:43:33.119528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:33.122920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:33.122920 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:33.141368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] MFA Rules not processed for user `921ba33e2e0740278b7b6fb4af248363`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:33.142559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:33.197473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1113/2200] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:43:36.072966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:43.190693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:43:56.113638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b990-5df3-4d23-bf7b-abd5a18c5764 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:43:56.114151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b990-5df3-4d23-bf7b-abd5a18c5764 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:43:56.114248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b990-5df3-4d23-bf7b-abd5a18c5764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:43:56.137709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-cd08b990-5df3-4d23-bf7b-abd5a18c5764 None None] MFA Rules not processed for user `eb0b112497dc4b64a9ea2f09f808646d`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:43:56.137709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cd08b990-5df3-4d23-bf7b-abd5a18c5764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:43:56.190932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1088/2201] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:05.982072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:05.983480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:06.051138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da63a0-9a9f-405c-b770-c15749e0d557 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:06.055683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:06.056341 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:06.056958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:06.056958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] PATH_INFO: `/v3/users/78aaa746883547099643e5eed3768343` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:06.058057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78aaa746883547099643e5eed3768343)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:06.060015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:06.183289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=25 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:06.277598 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78aaa746883547099643e5eed3768343'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:06.348340 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78aaa746883547099643e5eed3768343'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:06.363371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78aaa746883547099643e5eed3768343'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:06.364177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78aaa746883547099643e5eed3768343'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:06.364926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78aaa746883547099643e5eed3768343'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:06.383561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1114/2202] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:44:05 2022] DELETE /identity/v3/users/78aaa746883547099643e5eed3768343 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:06.388083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:06.388366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:06.388637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:06.408486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:06.409791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:06.474557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1089/2203] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:44:06.484962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:06.486729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:06.545463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aca9ad-784c-4a40-ac79-1b76d14b0cc5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:06.549120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd1032-9575-45d4-a6c7-2d5a1a6e330a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:06.549716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1032-9575-45d4-a6c7-2d5a1a6e330a service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:06.549991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1032-9575-45d4-a6c7-2d5a1a6e330a service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:06.550259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1032-9575-45d4-a6c7-2d5a1a6e330a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:06.559226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd1032-9575-45d4-a6c7-2d5a1a6e330a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:06.560634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd1032-9575-45d4-a6c7-2d5a1a6e330a service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:06.570423 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1115/2204] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:07.300049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:07.304756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:07.314339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76124-9ed0-480d-aff4-a0123b3eb378 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:07.319743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:07.320909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:07.320909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:07.321151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] PATH_INFO: `/v3/projects/266a4c8292344dc7989c968a30428354` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:07.322213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=266a4c8292344dc7989c968a30428354)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:07.324499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:07.430158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266a4c8292344dc7989c968a30428354', 'request_id': 'req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:07.437776 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266a4c8292344dc7989c968a30428354', 'request_id': 'req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:07.444132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266a4c8292344dc7989c968a30428354', 'request_id': 'req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:07.545868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] The token cache is being invalidate because project 266a4c8292344dc7989c968a30428354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:44:07.546152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:07.547428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:07.561054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1090/2205] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:44:07 2022] DELETE /identity/v3/projects/266a4c8292344dc7989c968a30428354 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:16.566271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:17.542330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:18.419222 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:18.419717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:18.419952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:18.440299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:18.441653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:18.501671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1116/2206] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:18.630545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:18.631974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:18.703216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d3e2b-ae65-492d-ad18-6b956e5aa5f4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:18.708077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5a6f3-91cb-464d-b0d7-c43dfca7058d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:18.708678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5a6f3-91cb-464d-b0d7-c43dfca7058d service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:18.708950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5a6f3-91cb-464d-b0d7-c43dfca7058d service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:18.709261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5a6f3-91cb-464d-b0d7-c43dfca7058d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:18.717101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5a6f3-91cb-464d-b0d7-c43dfca7058d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:18.718762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5a6f3-91cb-464d-b0d7-c43dfca7058d service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:18.726867 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1091/2207] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:44:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:44:20.511784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:20.513385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:20.575306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99f9d3-5a09-47a1-ab91-a066a0c6297e None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:20.580194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:20.580823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:20.581150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:20.581396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:20.583188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:20.658979 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:20.660534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0222cb-7de7-4961-8dcf-79c09aee95ac service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:20.672735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1117/2208] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:23.567480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b8f1e-d5ac-43af-8994-f821837f6464 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:23.568493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b8f1e-d5ac-43af-8994-f821837f6464 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:23.569108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b8f1e-d5ac-43af-8994-f821837f6464 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:23.571597 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1092/2209] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:44:23.586017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:23.586595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:23.586708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:23.607617 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] MFA Rules not processed for user `e7a0aa7593164a57970b48b14c14a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:23.609117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:23.655324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:23.691218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1118/2210] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:23.700538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b8f1e-d5ac-43af-8994-f821837f6464 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:23.703454 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-fa4b8f1e-d5ac-43af-8994-f821837f6464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:23.729986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:23.731729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:23.738713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b8f1e-d5ac-43af-8994-f821837f6464 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:23.745723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:23.746691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:23.747132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:23.747402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:23.758939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:23.760650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:23.775902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1093/2211] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:23.799284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09328c-7bed-4748-8e4f-0c5b2dc6ca9f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:23.803715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:23.803715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:23.804004 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:23.804190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:23.806459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:23.830914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:23.832371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:23.845082 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1119/2212] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:44:27.462460 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:27.463933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:27.536000 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc4478-0140-4e83-83ec-0d295208688c service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:27.540685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:27.541356 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:27.541632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:27.541902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] PATH_INFO: `/v3/users/277cc06a6df04ca4ab453de57ee84279` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:27.543022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=277cc06a6df04ca4ab453de57ee84279)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:27.544946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:27.757389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '277cc06a6df04ca4ab453de57ee84279'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:27.769646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '277cc06a6df04ca4ab453de57ee84279'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:27.850513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '277cc06a6df04ca4ab453de57ee84279'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:27.851129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '277cc06a6df04ca4ab453de57ee84279'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:27.851822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '277cc06a6df04ca4ab453de57ee84279'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:27.859130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:27.866807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1094/2213] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:44:27 2022] DELETE /identity/v3/users/277cc06a6df04ca4ab453de57ee84279 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:28.174403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:28.201669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5797-5ddf-4d4f-885a-df7e1aa17f03 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:28.318473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:28.319382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:28.319730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:28.320164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] PATH_INFO: `/v3/projects/e60a52ce75b94198ac6abb7727e7072d` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:28.321446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60a52ce75b94198ac6abb7727e7072d)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:28.326586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:28.438481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60a52ce75b94198ac6abb7727e7072d', 'request_id': 'req-c7293c6b-fa13-4521-a326-6f5a62a2abf8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.445361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60a52ce75b94198ac6abb7727e7072d', 'request_id': 'req-c7293c6b-fa13-4521-a326-6f5a62a2abf8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.519436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60a52ce75b94198ac6abb7727e7072d', 'request_id': 'req-c7293c6b-fa13-4521-a326-6f5a62a2abf8'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.531719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] The token cache is being invalidate because project e60a52ce75b94198ac6abb7727e7072d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:44:28.532121 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.532772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.534309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1120/2214] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:44:28 2022] DELETE /identity/v3/projects/e60a52ce75b94198ac6abb7727e7072d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:28.538345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:28.539814 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:28.553277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776815c-9bb0-46a0-977f-fbb87ba90ba0 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:28.621386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:28.622256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:28.622538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:28.622815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] PATH_INFO: `/v3/users/7817e638db8342d697761e7ccaaf1026` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:28.624579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7817e638db8342d697761e7ccaaf1026)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:28.627209 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:28.843148 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7817e638db8342d697761e7ccaaf1026'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.855538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7817e638db8342d697761e7ccaaf1026'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.923282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7817e638db8342d697761e7ccaaf1026'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.923913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7817e638db8342d697761e7ccaaf1026'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.924509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-82f768bb-5221-403a-81b8-e0d760c7361a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7817e638db8342d697761e7ccaaf1026'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:28.939238 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1095/2215] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:44:28 2022] DELETE /identity/v3/users/7817e638db8342d697761e7ccaaf1026 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:44:29.218610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:29.220280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:29.233718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7293c6b-fa13-4521-a326-6f5a62a2abf8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:29.247026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:29.299806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:29.300522 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:29.300828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:29.301140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] PATH_INFO: `/v3/projects/60b28b9d81b24b5b99dc2152516f4c2c` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:29.302327 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b28b9d81b24b5b99dc2152516f4c2c)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:29.304278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:29.423026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b28b9d81b24b5b99dc2152516f4c2c', 'request_id': 'req-13b42797-3b2b-4516-9bb2-a9f9c570ef88'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:29.429593 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b28b9d81b24b5b99dc2152516f4c2c', 'request_id': 'req-13b42797-3b2b-4516-9bb2-a9f9c570ef88'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:29.519156 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b28b9d81b24b5b99dc2152516f4c2c', 'request_id': 'req-13b42797-3b2b-4516-9bb2-a9f9c570ef88'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:29.530621 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] The token cache is being invalidate because project 60b28b9d81b24b5b99dc2152516f4c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:44:29.531021 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:29.531650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-13b42797-3b2b-4516-9bb2-a9f9c570ef88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:29.533157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1121/2216] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:44:29 2022] DELETE /identity/v3/projects/60b28b9d81b24b5b99dc2152516f4c2c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:29.538239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:29.538522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:29.539086 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:29.566874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:29.568336 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:29.635031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1096/2217] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:29.644217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:29.644490 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:29.644761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:29.664995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:29.666595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:29.674741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1122/2218] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:29.678886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:29.688717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aec42a-6f9e-4dad-93d7-166627d2c6e6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:29.693354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:29.693960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:29.694229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:29.694527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:29.695169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:44:29.695466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:29.697080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:29.709232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1097/2219] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:44:29.713637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:29.724260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22de87-a952-47d7-9851-0f534d2cb1a3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:29.729143 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:29.729745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:29.730027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:29.730299 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:29.730942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:29.732861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:29.848728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1123/2220] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:44:29 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:44:29.849976 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:29.856013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:29.864108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875f72-a3a4-4a09-b0e7-db9f4ab2555d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:29.923485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:29.924093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:29.924371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:29.924642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:29.925359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:29.927220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:30.038315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1098/2221] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:44:29 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:44:30.042464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:30.060140 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeec04-28f6-40e4-91f7-df6473cb4fed admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:30.065332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:30.065974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:30.066401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:30.066677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:30.067506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:30.069128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:30.081039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1124/2222] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:44:30.089632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:30.100275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cb743-0232-45eb-8b08-b92669028b43 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:30.105181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74db0a-7476-4100-befb-18bea919c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:30.105786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74db0a-7476-4100-befb-18bea919c478 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:30.106068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74db0a-7476-4100-befb-18bea919c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:30.106345 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74db0a-7476-4100-befb-18bea919c478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:30.106975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74db0a-7476-4100-befb-18bea919c478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:30.108644 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74db0a-7476-4100-befb-18bea919c478 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:30.119836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1099/2223] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:44:30.124653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:30.134907 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa271c-25da-41b2-b6e4-d5b57eba00a6 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:30.139534 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:30.141028 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:30.144011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:30.144011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] PATH_INFO: `/v3/projects/990b7e753ad1450489668d0ca151736e/users/6f0516ac4c6648f9b208a89e174da3a3/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:30.157795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] RBAC: Authorizing `identity:create_grant(project_id=990b7e753ad1450489668d0ca151736e, user_id=6f0516ac4c6648f9b208a89e174da3a3, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:30.160437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:30.229148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1125/2224] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:44:30 2022] PUT /identity/v3/projects/990b7e753ad1450489668d0ca151736e/users/6f0516ac4c6648f9b208a89e174da3a3/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:30.234441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:30.234724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:30.237606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:30.254632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:30.256409 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:30.323203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1100/2225] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:30.333814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:30.335678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:30.389371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:30.404406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d38518-376f-49ff-b0ff-9545a363988b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:30.409172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5192a4b-11c4-48cd-8e70-7087cd8dad18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:30.409794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5192a4b-11c4-48cd-8e70-7087cd8dad18 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:30.410083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5192a4b-11c4-48cd-8e70-7087cd8dad18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:30.410360 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5192a4b-11c4-48cd-8e70-7087cd8dad18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:30.420023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5192a4b-11c4-48cd-8e70-7087cd8dad18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:30.422276 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5192a4b-11c4-48cd-8e70-7087cd8dad18 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:30.434275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1126/2226] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:32.194144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:32.195889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:32.265563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58b989-810e-4ff8-bfb8-15a037d187ae None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:32.269228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:32.269879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:32.270159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:32.270452 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:32.272242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:32.315267 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:32.351474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:32.352928 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:32.366850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1101/2227] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:36.429885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:36.430455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:36.430455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:36.461306 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] MFA Rules not processed for user `6f0516ac4c6648f9b208a89e174da3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:36.462681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:36.539896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1127/2228] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:36.544551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:36.556942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a99af-f2de-4589-9d1d-497a03efc945 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:36.562182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:36.563215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:36.563501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:36.563772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:36.564404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:36.566396 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:36.653520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1102/2229] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:44:36 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:44:36.668474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:36.682583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f888bc-bcfb-446a-8f7a-0424660e92a3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:36.768825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:36.769619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:36.769930 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:36.770205 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:36.770848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:36.772824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:36.902063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:36.903679 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1128/2230] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:44:36 2022] POST /identity/v3/users => generated 295 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:44:36.916381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a848983-d7cc-415a-bf5f-e973d16b95b5 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:36.923252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:36.923252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:36.924296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:36.924296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:36.924736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:36.926377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:36.946521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1103/2231] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:44:36.952875 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:36.976615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a5790-e720-4f6e-96a7-3b0b5c7be843 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:36.986332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:36.987790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:36.988273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:36.988552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] PATH_INFO: `/v3/projects/aa384131227c4307945d5b791eb17e8d/users/6144492748a14024b7bbf7bfaac0d36e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:37.011972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa384131227c4307945d5b791eb17e8d, user_id=6144492748a14024b7bbf7bfaac0d36e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:37.014706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:37.086344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1129/2232] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:44:36 2022] PUT /identity/v3/projects/aa384131227c4307945d5b791eb17e8d/users/6144492748a14024b7bbf7bfaac0d36e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:37.090227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:37.106191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddc5a8-13af-4155-82f7-b096b05fbb6e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:37.192235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:37.193363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:37.193746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:37.194118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:37.194863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:37.197305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:37.213362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1104/2233] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:44:37.217482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:37.231109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb4b9-eb0c-4212-bb2c-1db0afa683df admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:37.236820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7b6e-93f4-4122-b6a0-c23685484ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:37.238967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7b6e-93f4-4122-b6a0-c23685484ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:37.238967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7b6e-93f4-4122-b6a0-c23685484ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:37.238967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7b6e-93f4-4122-b6a0-c23685484ffb admin admin] PATH_INFO: `/v3/projects/aa384131227c4307945d5b791eb17e8d/users/6144492748a14024b7bbf7bfaac0d36e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:37.240960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e7b6e-93f4-4122-b6a0-c23685484ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa384131227c4307945d5b791eb17e8d, user_id=6144492748a14024b7bbf7bfaac0d36e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:37.243861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e7b6e-93f4-4122-b6a0-c23685484ffb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:37.262071 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1130/2234] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:44:37 2022] PUT /identity/v3/projects/aa384131227c4307945d5b791eb17e8d/users/6144492748a14024b7bbf7bfaac0d36e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:43.045047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:43.048826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:43.224855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:43.225432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:43.225432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:43.235746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020d297-1915-4dd2-a5b0-9a3fe16e1fd1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:43.262761 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] MFA Rules not processed for user `6144492748a14024b7bbf7bfaac0d36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:43.263999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:43.299654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:43.300487 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:43.336677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:43.337501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:43.337857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:43.338194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:43.340875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:43.370159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1131/2235] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:44:43.380093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:43.381721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:43.412234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc643726-1133-4bfd-9df7-c5cdfc2bd9b3 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:43.421708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:43.422951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:43.423308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:43.423699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:43.463072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:43.465375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:43.489618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1105/2236] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:43.517552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:43.519532 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:43.549030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1132/2237] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:45.419483 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:45.420992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:45.486121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3dbe5-a9c9-4530-8b60-c55764e22bd8 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:45.490685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:45.491296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:45.491574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:45.491854 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:45.494170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:45.565268 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:45.566752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:45.589849 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1106/2238] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:49.955804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:49.957451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:49.973325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff574ab-875a-4657-b3ab-ddbf28f6be5e service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:49.978041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:49.978673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:49.978982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:49.979258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] PATH_INFO: `/v3/users/921ba33e2e0740278b7b6fb4af248363` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:49.980404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=921ba33e2e0740278b7b6fb4af248363)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:49.982396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:50.197877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921ba33e2e0740278b7b6fb4af248363'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.288630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921ba33e2e0740278b7b6fb4af248363'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.304204 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921ba33e2e0740278b7b6fb4af248363'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.304967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921ba33e2e0740278b7b6fb4af248363'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.305628 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-41f8d2d3-42d7-49f6-b870-c3bae5318d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921ba33e2e0740278b7b6fb4af248363'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.320912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1133/2239] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:44:49 2022] DELETE /identity/v3/users/921ba33e2e0740278b7b6fb4af248363 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:50.626578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:50.654021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e90ad-9854-4bb6-b5f7-26a022352ce3 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:50.685494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:50.740139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:50.740773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:50.741077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:50.741350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] PATH_INFO: `/v3/projects/edc675d4a92f4ff9ae8a41a020fa792c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:50.742484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc675d4a92f4ff9ae8a41a020fa792c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:50.744410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:50.881466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc675d4a92f4ff9ae8a41a020fa792c', 'request_id': 'req-786e215f-83db-4279-ba05-73c355032557'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.891473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc675d4a92f4ff9ae8a41a020fa792c', 'request_id': 'req-786e215f-83db-4279-ba05-73c355032557'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.899615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc675d4a92f4ff9ae8a41a020fa792c', 'request_id': 'req-786e215f-83db-4279-ba05-73c355032557'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.986047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] The token cache is being invalidate because project edc675d4a92f4ff9ae8a41a020fa792c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:44:50.986365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.988041 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-786e215f-83db-4279-ba05-73c355032557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:44:50.989239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1107/2240] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:44:50 2022] DELETE /identity/v3/projects/edc675d4a92f4ff9ae8a41a020fa792c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:50.994997 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397be07-ba93-4434-b239-9397f29331cb None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:50.995596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397be07-ba93-4434-b239-9397f29331cb None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:50.995596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397be07-ba93-4434-b239-9397f29331cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.015922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c397be07-ba93-4434-b239-9397f29331cb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:51.017070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c397be07-ba93-4434-b239-9397f29331cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:51.036315 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:51.088789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1134/2241] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:51.097560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.100019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.100555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.121764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:51.122960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:51.132562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1108/2242] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:51.137083 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397be07-ba93-4434-b239-9397f29331cb None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.148218 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397be07-ba93-4434-b239-9397f29331cb None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.152921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.153563 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.153845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.154117 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.154770 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:44:51.155148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.156864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.169237 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1135/2243] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:44:51.173865 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.183256 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6ca9-4396-4ee1-9ea4-08324e6c6d5d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.188489 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.189116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.189394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.189663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.190292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.192200 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.299571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1109/2244] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:44:51 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:44:51.300657 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.310809 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e9d83-49d4-417b-8911-a39621924ea0 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.370387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.371107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.371389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.371662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.372391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.375349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.487605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1136/2245] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:44:51 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:44:51.491903 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.502955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c886c-6560-4ce3-a331-b63319e810c8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.508343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.508971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.512757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.512757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.512757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.512757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.522729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1110/2246] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:44:51.526923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.537230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8c9ce-ac84-4bcf-aa94-4bbc8d1863b6 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.541844 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4b85-8c62-43db-8bbc-d6d4408de8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.542438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4b85-8c62-43db-8bbc-d6d4408de8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.542718 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4b85-8c62-43db-8bbc-d6d4408de8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.542992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4b85-8c62-43db-8bbc-d6d4408de8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.543633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4b85-8c62-43db-8bbc-d6d4408de8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.545260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4b85-8c62-43db-8bbc-d6d4408de8e8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.557056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1137/2247] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:44:51.561993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.572114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0bcc-6531-4c92-9171-7d02698a4fad admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.576677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.577406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.577761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.578150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] PATH_INFO: `/v3/projects/2bf7381b1bf545c39936c96595aa36f4/users/6069bdb19c5c42ff8546aef3197a6a2b/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.593643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf7381b1bf545c39936c96595aa36f4, user_id=6069bdb19c5c42ff8546aef3197a6a2b, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.596382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.657672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1111/2248] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:44:51 2022] PUT /identity/v3/projects/2bf7381b1bf545c39936c96595aa36f4/users/6069bdb19c5c42ff8546aef3197a6a2b/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:44:51.662828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a60d7-a73f-4c79-b4f4-7f0feadf09b7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.663118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a60d7-a73f-4c79-b4f4-7f0feadf09b7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.663392 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a60d7-a73f-4c79-b4f4-7f0feadf09b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.683147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-d42a60d7-a73f-4c79-b4f4-7f0feadf09b7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:51.684358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d42a60d7-a73f-4c79-b4f4-7f0feadf09b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:51.747101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1138/2249] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:51.758721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:51.760510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:51.818885 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cc17c-b750-4cd4-af20-54109ce009f8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:51.822477 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:51.823077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:51.823350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:51.823615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:51.829357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:51.832860 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:51.834316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:51.845109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1112/2250] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:44:57.392980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c798-864a-4ad2-9de2-35691f4e286c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:57.394669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c798-864a-4ad2-9de2-35691f4e286c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:57.394669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c798-864a-4ad2-9de2-35691f4e286c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:57.428259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3c23c798-864a-4ad2-9de2-35691f4e286c None None] MFA Rules not processed for user `6069bdb19c5c42ff8546aef3197a6a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:44:57.429797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3c23c798-864a-4ad2-9de2-35691f4e286c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:57.450691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:44:57.507824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1139/2251] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:44:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:44:57.523271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:44:57.524778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:44:57.606188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed4f82-9164-44ad-88f0-a49a6162e145 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:44:57.614070 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cf802-5604-4f15-8003-54b1a49fe75e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:44:57.614697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cf802-5604-4f15-8003-54b1a49fe75e service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:44:57.615027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cf802-5604-4f15-8003-54b1a49fe75e service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:44:57.615298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cf802-5604-4f15-8003-54b1a49fe75e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:44:57.625076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cf802-5604-4f15-8003-54b1a49fe75e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:44:57.626784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cf802-5604-4f15-8003-54b1a49fe75e service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:44:57.637118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1113/2252] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:00.211412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:00.211814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:00.211955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:00.243161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] MFA Rules not processed for user `6069bdb19c5c42ff8546aef3197a6a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:45:00.244722 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:00.254363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1140/2253] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:45:00.358658 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:00.360159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:00.360675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:00.390453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:45:00.391971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:00.470737 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1114/2254] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:45:00.483175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:00.485018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:00.584246 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f9df-0df4-4d10-8653-93fe073f8d3b None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:00.586304 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:00.586900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:00.587338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:00.587674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:00.598198 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:00.608834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:00.639000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1141/2255] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:45:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 14:45:00.698617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:00.698617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:00.703263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a68b74-36ae-46a1-ad19-0455194bbbcd None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:00.714253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:00.715016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:00.715290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:00.719337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:00.722179 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:00.744884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:00.746374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84384f-a0c7-4392-a94e-01bfd4186ba8 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:00.756586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1115/2256] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:01.562387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:01.565555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:01.639677 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6be34-70a5-4264-a362-0bcd8e91ea32 service swift] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:01.644577 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:01.645214 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:01.645495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:01.645766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:01.647485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:01.659228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:01.660804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:01.670829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1142/2257] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:02.812922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:02.814124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:02.814557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:02.838507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] MFA Rules not processed for user `6f0516ac4c6648f9b208a89e174da3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:45:02.844904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:02.940538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1116/2258] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:45:03.370086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:03.382310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f9df9-3cdc-4451-9226-6251340fd74d service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:03.386481 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:03.387368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:03.387753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:03.388085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:03.390669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:03.406540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:03.408380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:03.418264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1143/2259] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:05.818848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:05.833817 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c6a33-fca2-4cde-aa9c-2f0d16f0557d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:05.838288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:05.841713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:05.841713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:05.841713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:05.842380 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:05.856128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:05.858374 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:05.878870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1117/2260] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:07.247446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:07.252438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:07.284955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e64f56-9523-4d7b-9478-d9e929c025bc service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:07.308180 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:07.308951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:07.309362 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:07.309750 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:07.312452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:07.329764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:07.331492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:07.348041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1144/2261] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:10.933848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:10.943129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:45:10.959314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221295-770b-4288-aaba-625146142f43 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:10.966316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:10.967280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:10.967931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:10.968217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:10.971410 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:10.997122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:10.999680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2a50d-55e6-46e8-a50e-b5cce3f6aa66 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:11.020630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1118/2262] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:14.766922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:14.778726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:45:14.792003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736f1d6-7ad6-4f64-beba-c0dc27799107 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:14.795864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:14.796688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:14.797110 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:14.797474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:14.799649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:14.827485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:14.829321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:14.849479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1145/2263] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:21.074166 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:45:22.518824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:22.519421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:22.519421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:22.551459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:45:22.552953 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:22.561466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1119/2264] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:45:22.583873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:22.595641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8627400-8b54-442b-9e82-faa2d2f8f314 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:22.601212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff70a6-af40-4506-9b10-a6707c0c6ffb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:22.601855 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff70a6-af40-4506-9b10-a6707c0c6ffb service swift] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:22.602133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff70a6-af40-4506-9b10-a6707c0c6ffb service swift] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:22.602403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff70a6-af40-4506-9b10-a6707c0c6ffb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:22.611745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff70a6-af40-4506-9b10-a6707c0c6ffb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:22.613641 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff70a6-af40-4506-9b10-a6707c0c6ffb service swift] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:22.623812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1146/2265] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:45:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:45:32.546275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:45:32.618731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=10 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:45:34.796763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:34.806177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c838cd9-6738-4b27-8f71-1365d5b224ce None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:34.809842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:34.810442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:34.811213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:34.811213 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:34.815821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:34.880227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:34.881709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:34.891407 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1120/2266] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:45:40.201703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:40.202353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:40.202353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:40.222391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] MFA Rules not processed for user `c04dc4fcbdd6447daf928b3f14481011`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:45:40.223603 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:45:40.230701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1147/2267] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:45:40.243195 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:45:40.254360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a169c-03b8-4985-973b-bd98237e50ef service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:45:40.259583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0435bede480b4f83aee82223ef80836f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:45:40.260188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:45:40.260465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:45:40.260736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:45:40.269463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:45:40.271044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:45:40.279783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1121/2268] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 14:45:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 14:45:50.217286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:45:50.275900 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=36 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:46:23.306832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:23.343494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9d568-0b38-4c92-9d44-93fd08a1872a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:23.347118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:23.347715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:23.347992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:23.348260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:23.350009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:23.416163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:23.417648 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:23.438723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1148/2269] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:46:27.269280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:27.270063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:27.336161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e9ddd-0ffb-4757-9ab2-b130ce28b2bf service swift] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:27.343139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:27.343801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:27.344078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:27.344497 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:27.346519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:27.359211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:27.361807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:27.373500 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1122/2270] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:46:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:46:27.432331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:27.433832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:27.446513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebb254-276c-4d08-b575-293aac9feba8 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:27.452160 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:27.453623 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:27.453910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:27.454186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] PATH_INFO: `/v3/users/eb0b112497dc4b64a9ea2f09f808646d` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:27.455454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0b112497dc4b64a9ea2f09f808646d)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:27.457503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:27.648605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0b112497dc4b64a9ea2f09f808646d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:27.719789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0b112497dc4b64a9ea2f09f808646d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:27.733877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0b112497dc4b64a9ea2f09f808646d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:27.734624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0b112497dc4b64a9ea2f09f808646d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:27.735232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a5867513-9b31-454e-bc82-7f47f3a8d03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0b112497dc4b64a9ea2f09f808646d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:27.748703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1149/2271] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:46:27 2022] DELETE /identity/v3/users/eb0b112497dc4b64a9ea2f09f808646d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:46:28.007978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.019722 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30dc0-7895-42f8-ac35-22d377279757 service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.082548 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.083228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.083508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.083780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] PATH_INFO: `/v3/projects/0dd67ec4f08d4d548af5b5ccee50407c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.084934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd67ec4f08d4d548af5b5ccee50407c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:28.086912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:28.221788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd67ec4f08d4d548af5b5ccee50407c', 'request_id': 'req-da92d220-d881-4383-809e-c9e65d225dbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:28.228936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd67ec4f08d4d548af5b5ccee50407c', 'request_id': 'req-da92d220-d881-4383-809e-c9e65d225dbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:28.236322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd67ec4f08d4d548af5b5ccee50407c', 'request_id': 'req-da92d220-d881-4383-809e-c9e65d225dbe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:28.337063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] The token cache is being invalidate because project 0dd67ec4f08d4d548af5b5ccee50407c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:46:28.337691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:28.337888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-da92d220-d881-4383-809e-c9e65d225dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:46:28.339382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1123/2272] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:46:28 2022] DELETE /identity/v3/projects/0dd67ec4f08d4d548af5b5ccee50407c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:46:28.345067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.345352 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.345618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.400031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:46:28.408986 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:46:28.410211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:28.479688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1150/2273] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:46:28.488560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.488842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.489155 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.510405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:46:28.513889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:28.519850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1124/2274] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:46:28.524232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.535781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbcbbd-f5ce-436b-b10c-06b48e69a5e4 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.540784 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.541680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.542031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.544331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.545717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:46:28.549394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:28.549394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:28.560338 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1151/2275] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:46:28.564969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.574502 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae403-aab8-4222-880d-537ac448ec6e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.579439 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.580047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.580326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.580598 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.581275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:28.583185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:28.705123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1125/2276] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:46:28 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:46:28.709395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.720200 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0279-38e1-423e-b10b-18a70855350c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.779512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.780119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.780397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.780666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.781335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:28.783252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:28.892606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1152/2277] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:46:28 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:46:28.896794 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.908648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09145afb-f28f-42a2-8dcc-80bee08addd1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.913488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.914077 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.914353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.914630 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.915326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:28.916944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:28.919654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:46:28.928429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1126/2278] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:46:28.932445 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.943170 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f14281-eb97-4c35-bdb0-1be2518844a8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.947918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17941898-bc89-48f6-9a74-5dc0f08b5daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.948527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17941898-bc89-48f6-9a74-5dc0f08b5daf admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.948802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17941898-bc89-48f6-9a74-5dc0f08b5daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.949103 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17941898-bc89-48f6-9a74-5dc0f08b5daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:28.950244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17941898-bc89-48f6-9a74-5dc0f08b5daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:28.951883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17941898-bc89-48f6-9a74-5dc0f08b5daf admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:28.964636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1153/2279] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:46:28.978557 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:28.990108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcf445-44f2-4933-bcf5-6c52ad57bb9d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:28.994861 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:28.995512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:28.995791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:28.996063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] PATH_INFO: `/v3/projects/e428f318654f446db48445e4570ae2d3/users/7fa761a568644a9399d825498a51f054/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:29.013232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e428f318654f446db48445e4570ae2d3, user_id=7fa761a568644a9399d825498a51f054, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:29.016008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:29.104146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1127/2280] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:46:28 2022] PUT /identity/v3/projects/e428f318654f446db48445e4570ae2d3/users/7fa761a568644a9399d825498a51f054/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:46:29.109836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f903d0-088a-4d17-a945-e256a08fdbc9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:29.110122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f903d0-088a-4d17-a945-e256a08fdbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:29.110396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f903d0-088a-4d17-a945-e256a08fdbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:29.130608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-17f903d0-088a-4d17-a945-e256a08fdbc9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:46:29.131823 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-17f903d0-088a-4d17-a945-e256a08fdbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:29.191096 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1154/2281] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:46:29.201457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:29.203258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:29.274670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a8323-69c1-4da7-8702-64294d1705cb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:29.278437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:29.279264 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:29.279538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:29.279806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:29.289231 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:29.290668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:29.300405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1128/2282] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:46:35.118287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f62b8-0878-4e32-85c2-11eb28383aad None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:35.118833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f62b8-0878-4e32-85c2-11eb28383aad None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:35.118833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f62b8-0878-4e32-85c2-11eb28383aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:35.148184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:46:35.157119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ae1f62b8-0878-4e32-85c2-11eb28383aad None None] MFA Rules not processed for user `7fa761a568644a9399d825498a51f054`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:46:35.162826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ae1f62b8-0878-4e32-85c2-11eb28383aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:35.226113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1155/2283] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:46:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:46:35.243388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:46:35.245046 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:35.330810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509bc20-dfb2-4cc5-ad4d-d6a930a76714 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:46:35.336767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a0312-cf1d-4cac-be2a-64b678a2fbec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:46:35.337734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a0312-cf1d-4cac-be2a-64b678a2fbec service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:35.338082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a0312-cf1d-4cac-be2a-64b678a2fbec service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:35.338424 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a0312-cf1d-4cac-be2a-64b678a2fbec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:35.348027 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a0312-cf1d-4cac-be2a-64b678a2fbec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:46:35.349694 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a0312-cf1d-4cac-be2a-64b678a2fbec service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:46:35.360710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1129/2284] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:46:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:46:38.129279 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bcccd-f2ca-4f6a-9b58-fd6bb4694283 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:46:38.129808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bcccd-f2ca-4f6a-9b58-fd6bb4694283 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:46:38.129808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bcccd-f2ca-4f6a-9b58-fd6bb4694283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:46:38.150172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-224bcccd-f2ca-4f6a-9b58-fd6bb4694283 None None] MFA Rules not processed for user `7fa761a568644a9399d825498a51f054`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:46:38.152134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-224bcccd-f2ca-4f6a-9b58-fd6bb4694283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:46:38.165906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1156/2285] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:46:45.354965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:46:48.144666 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=30 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:47:40.091468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:47:40.092088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:47:40.092088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:47:40.113528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] MFA Rules not processed for user `6f0516ac4c6648f9b208a89e174da3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:47:40.114762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:47:40.176412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1130/2286] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:47:42.892634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d9f9-a2fa-4065-a901-38efbcb74d44 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:47:42.893228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d9f9-a2fa-4065-a901-38efbcb74d44 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:47:42.893228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d9f9-a2fa-4065-a901-38efbcb74d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:47:42.913888 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-0637d9f9-a2fa-4065-a901-38efbcb74d44 None None] MFA Rules not processed for user `7fa761a568644a9399d825498a51f054`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:47:42.915151 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0637d9f9-a2fa-4065-a901-38efbcb74d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:47:42.922353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1157/2287] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:47:47.924001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:47:47.925656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:47:48.037828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869de16-3aa8-4fc2-bc19-4f857cbf72bb None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:47:48.043321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:47:48.043941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:47:48.044222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:47:48.044495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:47:48.046309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:47:48.059634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:47:48.061279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd8a59-ac86-46a5-b2b7-b2468b7988bf service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:47:48.073706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1131/2288] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:47:52.907970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:47:58.068843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=66 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:48:05.759477 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:48:05.760041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:48:05.760041 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:48:05.779924 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] MFA Rules not processed for user `6069bdb19c5c42ff8546aef3197a6a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:48:05.781317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:48:05.836535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1158/2289] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:48:15.829851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=25 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:48:53.591304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:48:53.591977 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:48:53.591977 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:48:53.613851 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] MFA Rules not processed for user `7fa761a568644a9399d825498a51f054`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:48:53.615580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:48:53.623879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1132/2290] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:48:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:03.607826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:24.818572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:24.830093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ee56c-6147-4501-9565-24ab36a170c7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:24.834014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:24.834681 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:24.834996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:24.835308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:24.837394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:24.875821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:24.877528 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:24.905138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1159/2291] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:49:26.110915 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:26.122688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd2209-ef81-4f96-8e56-f69d9bac784e None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:26.126504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:26.127121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:26.127394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:26.127663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:26.129551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:26.144318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:26.145798 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:26.162376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1133/2292] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:49:34.900671 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=18 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:36.152474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:39.171572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:39.173495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:39.192452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea238e7-8f31-4265-8780-42e1887136b4 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:39.200956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:39.200956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:39.200956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:39.200956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:39.202172 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:39.216389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:39.219596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:39.239861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1160/2293] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:49:39.320577 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:39.323120 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:39.347572 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154616b-ebcf-464a-8f4d-3c1019795141 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:39.353582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:39.354235 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:39.354516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:39.354778 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] PATH_INFO: `/v3/users/6069bdb19c5c42ff8546aef3197a6a2b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:39.355961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6069bdb19c5c42ff8546aef3197a6a2b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:39.358677 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:39.522328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6069bdb19c5c42ff8546aef3197a6a2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:39.547910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6069bdb19c5c42ff8546aef3197a6a2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:39.645369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6069bdb19c5c42ff8546aef3197a6a2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:39.646094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6069bdb19c5c42ff8546aef3197a6a2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:39.646774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-185fa2c1-1d31-4328-9b79-7983d9c25520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6069bdb19c5c42ff8546aef3197a6a2b'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:39.662959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1134/2294] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:49:39 2022] DELETE /identity/v3/users/6069bdb19c5c42ff8546aef3197a6a2b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:39.961897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:39.980326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a5f63-2019-48e8-9900-17b72e76201a service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.055378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.055932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.056213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.056487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] PATH_INFO: `/v3/projects/2bf7381b1bf545c39936c96595aa36f4` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.057730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf7381b1bf545c39936c96595aa36f4)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.060248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.175106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf7381b1bf545c39936c96595aa36f4', 'request_id': 'req-51e5befb-8c21-4131-9889-627444b6de91'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:40.187009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf7381b1bf545c39936c96595aa36f4', 'request_id': 'req-51e5befb-8c21-4131-9889-627444b6de91'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:40.263042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf7381b1bf545c39936c96595aa36f4', 'request_id': 'req-51e5befb-8c21-4131-9889-627444b6de91'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:40.275733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] The token cache is being invalidate because project 2bf7381b1bf545c39936c96595aa36f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:49:40.276039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:40.276704 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-51e5befb-8c21-4131-9889-627444b6de91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:40.278416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1161/2295] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:49:39 2022] DELETE /identity/v3/projects/2bf7381b1bf545c39936c96595aa36f4 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:49:40.283752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.284021 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.284290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.306154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:40.307509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:40.389941 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1135/2296] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:49:40.399169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.399583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.399981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.422910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:40.424335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:40.433437 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1162/2297] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:40.438804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:40.450555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68e1c7-d3bf-4200-9c9f-a262d1436515 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.455441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.456056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.456331 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.456599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.457293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:49:40.457591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.459229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.472946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1136/2298] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:49:40.478747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:40.489057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d8085-a8d8-447e-93e7-ee9da0c0441d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.494612 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.495260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.495533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.495801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.496450 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.499565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.598436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1163/2299] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:49:40 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:49:40.603275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:40.615035 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e821fc-1f43-4428-b8a9-6ade87ff2f6f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.685482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.686105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.686381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.686652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.687289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.689286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.698026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:40.780468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1137/2300] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:49:40 2022] POST /identity/v3/users => generated 303 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:49:40.786600 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:40.799351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd3dac-8d42-4f0e-8536-bd62e2cdc537 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.804581 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.805409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.805757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.806145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.806908 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.808582 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.821330 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1164/2301] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:40.826721 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:40.837315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d5134-0846-4bee-9dc3-bb19c9af8297 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.842228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5f63-f7b2-4204-87ac-35298a887b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.842838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5f63-f7b2-4204-87ac-35298a887b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.843110 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5f63-f7b2-4204-87ac-35298a887b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.843377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5f63-f7b2-4204-87ac-35298a887b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.844024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f5f63-f7b2-4204-87ac-35298a887b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.845795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f5f63-f7b2-4204-87ac-35298a887b24 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.858629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1138/2302] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:49:40.863254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:40.874866 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f83b0-b355-48a9-83be-108c383674d6 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:40.879706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:40.880378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:40.880654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:40.880923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] PATH_INFO: `/v3/projects/079d9b3743df4a13808284219026a1e5/users/79e9f54664ed4bcc9e49494a55007aeb/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:40.899686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079d9b3743df4a13808284219026a1e5, user_id=79e9f54664ed4bcc9e49494a55007aeb, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:40.902449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:40.997091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1165/2303] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:40 2022] PUT /identity/v3/projects/079d9b3743df4a13808284219026a1e5/users/79e9f54664ed4bcc9e49494a55007aeb/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:41.002895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9caa-9d75-4e01-a9a3-96b93906e270 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:41.003174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9caa-9d75-4e01-a9a3-96b93906e270 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:41.003478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9caa-9d75-4e01-a9a3-96b93906e270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:41.024959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-93fc9caa-9d75-4e01-a9a3-96b93906e270 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:41.026243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-93fc9caa-9d75-4e01-a9a3-96b93906e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:41.097375 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1139/2304] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:49:41.110321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:41.112105 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:41.129945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:41.185530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02cb22-be09-4a4f-9960-2b167348d0a3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:41.189743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:41.190697 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:41.191386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:41.191742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:41.203378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:41.204917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:41.217541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1166/2305] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:49:42.571641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:42.572555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:42.572555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:42.614187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] MFA Rules not processed for user `79e9f54664ed4bcc9e49494a55007aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:42.616098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:42.719799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1140/2306] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:42.721733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:42.737468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a53ed-c992-4b72-ac6c-8ec255940978 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:42.743271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:42.743991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:42.744312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:42.744687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:42.745558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:42.748090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:42.847688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1167/2307] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:49:42 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:49:42.850028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:42.872174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120f147-1caf-4deb-9d01-0e56c019d6b5 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:42.985707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:42.986533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:42.986913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:42.987324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:42.988404 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:42.991779 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:43.096850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:43.103319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1141/2308] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:49:42 2022] POST /identity/v3/users => generated 309 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:49:43.108561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:43.123687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde3ba5-2416-4a44-97e6-95197c85d49d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:43.129653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:43.136252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:43.136252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:43.136252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:43.136252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:43.136252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:43.146436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1168/2309] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:43.153706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:43.166624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ad668-5513-4a14-a711-0f79dbac0f97 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:43.183016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:43.184174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:43.184634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:43.185468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] PATH_INFO: `/v3/projects/5232d4b02d6c4525abd0cd27ce2d38ec/users/590a38c2d3364092b5d6c3a28cd3a0e5/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:43.212242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5232d4b02d6c4525abd0cd27ce2d38ec, user_id=590a38c2d3364092b5d6c3a28cd3a0e5, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:43.216593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:43.333859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1142/2310] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:43 2022] PUT /identity/v3/projects/5232d4b02d6c4525abd0cd27ce2d38ec/users/590a38c2d3364092b5d6c3a28cd3a0e5/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:43.338137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:43.349711 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fc263-2740-4904-9825-95bce1305e3e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:43.428654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:43.429457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:43.429852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:43.430308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:43.431073 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:43.432971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:43.450555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1169/2311] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:49:43.452239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:43.464440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512a26a-379c-49d7-bb1b-20c58f14f558 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:43.470453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bed5c-cb29-4244-b2c0-cae347073e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:43.471156 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bed5c-cb29-4244-b2c0-cae347073e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:43.471431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bed5c-cb29-4244-b2c0-cae347073e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:43.471709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bed5c-cb29-4244-b2c0-cae347073e14 admin admin] PATH_INFO: `/v3/projects/5232d4b02d6c4525abd0cd27ce2d38ec/users/590a38c2d3364092b5d6c3a28cd3a0e5/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:43.473756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bed5c-cb29-4244-b2c0-cae347073e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5232d4b02d6c4525abd0cd27ce2d38ec, user_id=590a38c2d3364092b5d6c3a28cd3a0e5, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:43.476427 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bed5c-cb29-4244-b2c0-cae347073e14 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:43.488524 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1143/2312] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:43 2022] PUT /identity/v3/projects/5232d4b02d6c4525abd0cd27ce2d38ec/users/590a38c2d3364092b5d6c3a28cd3a0e5/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:43.566747 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:43.569432 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:43.727354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ea10-2ef1-4f5b-b269-cdd4529bb24a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:43.752816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:43.815596 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:43.816658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:43.817393 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:43.818323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:43.822488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:43.947634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:43.950687 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:43.968228 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1170/2313] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:49:45.024852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47beb7-c54b-4dc2-b3dd-9ea00b5a4767 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:45.025539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47beb7-c54b-4dc2-b3dd-9ea00b5a4767 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:45.025739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47beb7-c54b-4dc2-b3dd-9ea00b5a4767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:45.072907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-ae47beb7-c54b-4dc2-b3dd-9ea00b5a4767 None None] MFA Rules not processed for user `590a38c2d3364092b5d6c3a28cd3a0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:45.075863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ae47beb7-c54b-4dc2-b3dd-9ea00b5a4767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:45.209585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1144/2314] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:47.246033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:47.357624 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e43d4-7590-491a-857b-d7d6201a7d78 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:47.363376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:47.364136 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:47.364502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:47.364904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] PATH_INFO: `/v3/users/79e9f54664ed4bcc9e49494a55007aeb` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:47.366551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e9f54664ed4bcc9e49494a55007aeb)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:47.370351 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:47.384006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:47.384006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:47.384006 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:47.410818 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] MFA Rules not processed for user `6f0516ac4c6648f9b208a89e174da3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:47.412639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:47.422364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1145/2315] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:49:47.614885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e9f54664ed4bcc9e49494a55007aeb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:47.704424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e9f54664ed4bcc9e49494a55007aeb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:47.723097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e9f54664ed4bcc9e49494a55007aeb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:47.723967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e9f54664ed4bcc9e49494a55007aeb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:47.724642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e9f54664ed4bcc9e49494a55007aeb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:47.726376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:47.757925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1171/2316] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:49:47 2022] DELETE /identity/v3/users/79e9f54664ed4bcc9e49494a55007aeb => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:48.103315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:48.119683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4d275-edb5-41a4-803b-d912f78e2d10 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:48.217463 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:48.218322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:48.218766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:48.219212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] PATH_INFO: `/v3/projects/079d9b3743df4a13808284219026a1e5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:48.221413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079d9b3743df4a13808284219026a1e5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:48.223971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:48.369239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:48.377634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079d9b3743df4a13808284219026a1e5', 'request_id': 'req-371af0b9-324e-4b45-8f74-f8f831ce0410'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.385056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079d9b3743df4a13808284219026a1e5', 'request_id': 'req-371af0b9-324e-4b45-8f74-f8f831ce0410'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.393103 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079d9b3743df4a13808284219026a1e5', 'request_id': 'req-371af0b9-324e-4b45-8f74-f8f831ce0410'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.492549 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] The token cache is being invalidate because project 079d9b3743df4a13808284219026a1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:49:48.492866 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.495305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.495444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1146/2317] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:49:48 2022] DELETE /identity/v3/projects/079d9b3743df4a13808284219026a1e5 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:49:48.500420 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:48.502039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:48.517145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebf115-5f8a-4f5a-bbaf-d760212838d5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:48.587308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:48.587955 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:48.588241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:48.588509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] PATH_INFO: `/v3/users/590a38c2d3364092b5d6c3a28cd3a0e5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:48.589693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590a38c2d3364092b5d6c3a28cd3a0e5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:48.591833 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:48.857434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590a38c2d3364092b5d6c3a28cd3a0e5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.950043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590a38c2d3364092b5d6c3a28cd3a0e5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.966862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590a38c2d3364092b5d6c3a28cd3a0e5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.967789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590a38c2d3364092b5d6c3a28cd3a0e5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.968400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1e185fe8-d249-4aab-9255-d965ea9f96a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590a38c2d3364092b5d6c3a28cd3a0e5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:48.985572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1172/2318] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:49:48 2022] DELETE /identity/v3/users/590a38c2d3364092b5d6c3a28cd3a0e5 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:49.306850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:49.314315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:49.328464 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371af0b9-324e-4b45-8f74-f8f831ce0410 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:49.414931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:49.415645 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:49.415766 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:49.415997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] PATH_INFO: `/v3/projects/5232d4b02d6c4525abd0cd27ce2d38ec` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:49.417202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5232d4b02d6c4525abd0cd27ce2d38ec)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:49.419222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:49.595053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5232d4b02d6c4525abd0cd27ce2d38ec', 'request_id': 'req-1885099f-a857-49a1-9753-9aaf9a1d41d8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:49.601848 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5232d4b02d6c4525abd0cd27ce2d38ec', 'request_id': 'req-1885099f-a857-49a1-9753-9aaf9a1d41d8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:49.608738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5232d4b02d6c4525abd0cd27ce2d38ec', 'request_id': 'req-1885099f-a857-49a1-9753-9aaf9a1d41d8'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:49.689649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] The token cache is being invalidate because project 5232d4b02d6c4525abd0cd27ce2d38ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:49:49.689874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:49.690676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1885099f-a857-49a1-9753-9aaf9a1d41d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:49:49.692679 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1147/2319] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:49:49 2022] DELETE /identity/v3/projects/5232d4b02d6c4525abd0cd27ce2d38ec => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:49.697790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:49.698467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:49.698467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:49.721818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:49.725353 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:49.787655 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1173/2320] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:49.796542 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:49.796841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:49.797130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:49.819627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:49.821108 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:49.829465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1148/2321] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:49.835023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:49.853786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efabf-b956-4de9-88c6-52321c12edfd None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:49.863455 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:49.864239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:49.864524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:49.865032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:49.866322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:49:49.866876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:49.870324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:49.876396 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:49.894580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1174/2322] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:49:49.899965 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:49.917153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38523ce-dfb3-4cdb-acae-7074b2021da6 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:49.927225 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:49.928347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:49.928843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:49.929445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:49.930659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:49.935912 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:50.054533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1149/2323] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:49:49 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:49:50.060599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:50.080649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37e680-678e-4846-bd4c-c2d71371ae7a admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:50.166561 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:50.167436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:50.167914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:50.168459 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:50.169891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:50.173504 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:50.289533 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1175/2324] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:49:50 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:49:50.295444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:50.310627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c8d1-ba1c-44b6-8d97-be633dc5f98d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:50.332149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:50.333475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:50.333911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:50.334287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:50.335478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:50.337805 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:50.359620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1150/2325] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:50.365046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:50.379113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468e56c-d95c-4e98-882b-94a35c0b9de8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:50.385830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c4778-2e0b-4ef2-b162-f64fbf9ee147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:50.386462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c4778-2e0b-4ef2-b162-f64fbf9ee147 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:50.386735 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c4778-2e0b-4ef2-b162-f64fbf9ee147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:50.387565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c4778-2e0b-4ef2-b162-f64fbf9ee147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:50.388230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c4778-2e0b-4ef2-b162-f64fbf9ee147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:50.389992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c4778-2e0b-4ef2-b162-f64fbf9ee147 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:50.402896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1176/2326] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:50.408417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:50.427255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f3f7a-b64b-4dcb-ba0a-61dad057f737 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:50.435194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:50.436252 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:50.436719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:50.437194 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] PATH_INFO: `/v3/projects/ffddc5c6d3954d54b95d35c529ecd91a/users/991c5eaf64694056912ede5e9d48a595/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:50.466324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffddc5c6d3954d54b95d35c529ecd91a, user_id=991c5eaf64694056912ede5e9d48a595, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:50.471528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:50.477116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:50.613876 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1151/2327] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:50 2022] PUT /identity/v3/projects/ffddc5c6d3954d54b95d35c529ecd91a/users/991c5eaf64694056912ede5e9d48a595/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:49:50.625781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:50.626075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:50.626334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:50.659391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] MFA Rules not processed for user `991c5eaf64694056912ede5e9d48a595`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:50.660625 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:50.754927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1177/2328] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:50.761757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:50.850888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777f46c-bf50-4404-80da-607e4042f13d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:50.857606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:50.858406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:50.858687 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:50.859001 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:50.859656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:50.861969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:50.999139 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1152/2329] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:49:50 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:49:51.003922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:51.023756 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eecea-8c79-4717-9101-b89b021faff7 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:51.096391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:51.097055 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.097331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.097606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.098263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:51.100278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:51.112776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:51.216138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1178/2330] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:49:51 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:49:51.219955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:51.231242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd435f0-aa1d-49fb-93d8-e110c8fa4177 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:51.236969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:51.237749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.238030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.238301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.238950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:51.240563 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:51.257301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1153/2331] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:49:51.264033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:51.279513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8bce9-f4b2-4fd1-98ed-7430c8a9e638 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:51.284777 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ca12c-3b41-4471-826a-fa6870a7cefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:51.285456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ca12c-3b41-4471-826a-fa6870a7cefe admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.285736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ca12c-3b41-4471-826a-fa6870a7cefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.286013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ca12c-3b41-4471-826a-fa6870a7cefe admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.286673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ca12c-3b41-4471-826a-fa6870a7cefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:51.288344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ca12c-3b41-4471-826a-fa6870a7cefe admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:51.301057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1179/2332] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:51.305733 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:51.316892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64887155-661f-46f0-9ab6-869ea3b201ab admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:51.322691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:51.323504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.323819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.324133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] PATH_INFO: `/v3/projects/f323a2574d7540289ce57963e5449cf0/users/bec4e7b7e1b442a088aba3f11a88b199/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.342930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f323a2574d7540289ce57963e5449cf0, user_id=bec4e7b7e1b442a088aba3f11a88b199, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:51.345675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:51.430373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1154/2333] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:51 2022] PUT /identity/v3/projects/f323a2574d7540289ce57963e5449cf0/users/bec4e7b7e1b442a088aba3f11a88b199/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:49:51.445548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.445959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.446305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.486077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] MFA Rules not processed for user `bec4e7b7e1b442a088aba3f11a88b199`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:51.488257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:51.619427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1180/2334] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:51.625968 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:51.724188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cb40e-cdf5-47d5-8dbf-f6c2665fec0e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:51.732972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:51.734495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.734986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.735342 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.736272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:51.740078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:51.843807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1155/2335] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:49:51 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:49:51.848145 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:51.860909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63c064-3434-4084-a57c-412c56c04565 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:51.937540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:51.938493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:51.938870 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:51.939254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:51.940031 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:51.942280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:52.025802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1181/2336] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:49:51 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:49:52.031073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:52.045041 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d8c5-e53e-4f45-9aee-fab140354ff8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:52.050582 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:52.052438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:52.052438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:52.052438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:52.052438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:52.054023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:52.066772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1156/2337] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:52.072917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:52.084856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a905c8-7ae9-4197-92c6-97a1ea534e7d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:52.090239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:52.092680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:52.092680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:52.092680 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] PATH_INFO: `/v3/projects/b5849686fe0d4b2d8a25c940d2086fa1/users/acdfd0d6040546ba988b565f2d97538e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:52.113367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5849686fe0d4b2d8a25c940d2086fa1, user_id=acdfd0d6040546ba988b565f2d97538e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:52.117434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:52.257184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1182/2338] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:52 2022] PUT /identity/v3/projects/b5849686fe0d4b2d8a25c940d2086fa1/users/acdfd0d6040546ba988b565f2d97538e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:49:52.264315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:52.297994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe39c3-6fae-4fe2-a5a0-97381c311107 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:52.319290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:52.367540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3516f09-265c-4728-ab36-c553be2517e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:52.368650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3516f09-265c-4728-ab36-c553be2517e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:52.368934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3516f09-265c-4728-ab36-c553be2517e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:52.369228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3516f09-265c-4728-ab36-c553be2517e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:52.369931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3516f09-265c-4728-ab36-c553be2517e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:52.372152 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3516f09-265c-4728-ab36-c553be2517e5 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:52.385324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1157/2339] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:49:52.390227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:52.400978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796a588-b693-4bb2-a5b5-7e14e32a2f98 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:52.406011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:52.406690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:52.406960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:52.407227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] PATH_INFO: `/v3/projects/b5849686fe0d4b2d8a25c940d2086fa1/users/acdfd0d6040546ba988b565f2d97538e/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:52.409254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5849686fe0d4b2d8a25c940d2086fa1, user_id=acdfd0d6040546ba988b565f2d97538e, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:52.411944 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:52.423720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1183/2340] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:49:52 2022] PUT /identity/v3/projects/b5849686fe0d4b2d8a25c940d2086fa1/users/acdfd0d6040546ba988b565f2d97538e/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:49:52.434659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:52.435499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:52.435499 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:52.476223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] MFA Rules not processed for user `acdfd0d6040546ba988b565f2d97538e`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:49:52.478018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:52.595525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1158/2341] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:49:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:49:52.610278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:52.613268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:52.664983 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:49:52.715491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3321bd3-5454-4561-9fa2-79ddd50ca39d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:52.720990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:52.720990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:52.720990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:52.720990 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:52.733508 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:52.735035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:52.749572 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1184/2342] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:49:54.156054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:54.158118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:49:54.174584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a74f5-751e-43f1-ba69-d6468d9311f8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:54.178678 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:54.179330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:54.179612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:54.179875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:54.260189 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:54.263600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:54.281940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1159/2343] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:49:59.465843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:49:59.475800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf207d5-12a2-45aa-9220-c25b3379afe0 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:49:59.479529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12414a-c5b4-4432-979a-eb3c37ad3403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:49:59.480146 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12414a-c5b4-4432-979a-eb3c37ad3403 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:49:59.480467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12414a-c5b4-4432-979a-eb3c37ad3403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:49:59.480730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12414a-c5b4-4432-979a-eb3c37ad3403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:49:59.555738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12414a-c5b4-4432-979a-eb3c37ad3403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:49:59.558570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12414a-c5b4-4432-979a-eb3c37ad3403 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:49:59.576235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1185/2344] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:49:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:04.253967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:04.303774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:04.326018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3d4a7-139f-4e01-a48a-38a5bd129cc1 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:04.330738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:04.331388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:04.331659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:04.332083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] PATH_INFO: `/v3/users/991c5eaf64694056912ede5e9d48a595` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:04.333383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991c5eaf64694056912ede5e9d48a595)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:04.335355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:04.536390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991c5eaf64694056912ede5e9d48a595'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:04.566104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991c5eaf64694056912ede5e9d48a595'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:04.662309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991c5eaf64694056912ede5e9d48a595'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:04.663583 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991c5eaf64694056912ede5e9d48a595'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:04.664888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991c5eaf64694056912ede5e9d48a595'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:04.704559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1160/2345] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:04 2022] DELETE /identity/v3/users/991c5eaf64694056912ede5e9d48a595 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:04.710277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:04.711744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:04.712447 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:04.747388 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:04.749435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:04.866767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1186/2346] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:04.879080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:04.880975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:04.989755 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c15a8-b5d6-4c79-b81e-216de0c5e690 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:04.996933 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:04.998127 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:04.998569 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:04.999144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:05.017632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:05.020590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:05.040745 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1161/2347] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:05.738383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:05.740674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:05.773698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126036ff-ab94-433a-a353-7ab980abe9eb None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:05.782027 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:05.783048 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:05.783554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:05.784003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] PATH_INFO: `/v3/projects/ffddc5c6d3954d54b95d35c529ecd91a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:05.785778 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffddc5c6d3954d54b95d35c529ecd91a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:05.787562 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:05.927749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffddc5c6d3954d54b95d35c529ecd91a', 'request_id': 'req-58fd7721-4f4e-4976-a3bb-e47e8555d415'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:05.935694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffddc5c6d3954d54b95d35c529ecd91a', 'request_id': 'req-58fd7721-4f4e-4976-a3bb-e47e8555d415'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.045046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffddc5c6d3954d54b95d35c529ecd91a', 'request_id': 'req-58fd7721-4f4e-4976-a3bb-e47e8555d415'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.046863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:06.059095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] The token cache is being invalidate because project ffddc5c6d3954d54b95d35c529ecd91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:06.059373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.060305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.062124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1187/2348] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:05 2022] DELETE /identity/v3/projects/ffddc5c6d3954d54b95d35c529ecd91a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:06.066346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:06.077676 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcd91-8706-4821-aaab-3df26b1cdeaa service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:06.147925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:06.148675 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:06.149048 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:06.149344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] PATH_INFO: `/v3/users/bec4e7b7e1b442a088aba3f11a88b199` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:06.150514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec4e7b7e1b442a088aba3f11a88b199)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:06.152790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:06.158608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:06.426364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec4e7b7e1b442a088aba3f11a88b199'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.442134 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec4e7b7e1b442a088aba3f11a88b199'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.526824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec4e7b7e1b442a088aba3f11a88b199'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.527820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec4e7b7e1b442a088aba3f11a88b199'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.528545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec4e7b7e1b442a088aba3f11a88b199'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:06.545137 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1162/2349] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:06 2022] DELETE /identity/v3/users/bec4e7b7e1b442a088aba3f11a88b199 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:07.058444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:07.060695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:07.076212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd7721-4f4e-4976-a3bb-e47e8555d415 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:07.183553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:07.184226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:07.184552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:07.184859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] PATH_INFO: `/v3/projects/f323a2574d7540289ce57963e5449cf0` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:07.186123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f323a2574d7540289ce57963e5449cf0)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:07.188099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:07.712226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f323a2574d7540289ce57963e5449cf0', 'request_id': 'req-81f7ca24-0c68-4a67-85d9-a53b4c01307e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:07.721914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f323a2574d7540289ce57963e5449cf0', 'request_id': 'req-81f7ca24-0c68-4a67-85d9-a53b4c01307e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:07.882651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f323a2574d7540289ce57963e5449cf0', 'request_id': 'req-81f7ca24-0c68-4a67-85d9-a53b4c01307e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:07.913821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] The token cache is being invalidate because project f323a2574d7540289ce57963e5449cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:07.914269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:07.915275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:07.918794 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1188/2350] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:07 2022] DELETE /identity/v3/projects/f323a2574d7540289ce57963e5449cf0 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:07.924846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:07.927214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:07.947017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a33b0a-da74-4ab6-aa7d-bc3c659502dd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:08.025271 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:08.025944 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:08.026220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:08.026488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] PATH_INFO: `/v3/users/acdfd0d6040546ba988b565f2d97538e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:08.028530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acdfd0d6040546ba988b565f2d97538e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:08.030637 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:08.196277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acdfd0d6040546ba988b565f2d97538e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:08.215416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acdfd0d6040546ba988b565f2d97538e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:08.292501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acdfd0d6040546ba988b565f2d97538e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:08.294769 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acdfd0d6040546ba988b565f2d97538e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:08.295531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acdfd0d6040546ba988b565f2d97538e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:08.307910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:08.314595 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1163/2351] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:07 2022] DELETE /identity/v3/users/acdfd0d6040546ba988b565f2d97538e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:09.327179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:09.327179 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:09.350962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7ca24-0c68-4a67-85d9-a53b4c01307e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:09.441114 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:09.443188 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:09.443678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:09.444166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] PATH_INFO: `/v3/projects/b5849686fe0d4b2d8a25c940d2086fa1` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:09.446268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5849686fe0d4b2d8a25c940d2086fa1)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:09.454861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:09.486139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:09.522321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:09.524354 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:09.595431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5849686fe0d4b2d8a25c940d2086fa1', 'request_id': 'req-0b937b41-8baf-49d4-bc3d-b61c99ef02db'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:09.604896 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5849686fe0d4b2d8a25c940d2086fa1', 'request_id': 'req-0b937b41-8baf-49d4-bc3d-b61c99ef02db'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:09.612668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf66b84-ee43-4403-b64d-da123d9d8481 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:09.668355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:09.669228 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:09.669624 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:09.670020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:09.672159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:09.683875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:09.685639 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02a4b-e228-4b77-af06-a8b284d2487b service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:09.694360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1164/2352] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:50:09.707387 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5849686fe0d4b2d8a25c940d2086fa1', 'request_id': 'req-0b937b41-8baf-49d4-bc3d-b61c99ef02db'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:09.730281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] The token cache is being invalidate because project b5849686fe0d4b2d8a25c940d2086fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:09.732885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:09.732885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0b937b41-8baf-49d4-bc3d-b61c99ef02db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:09.734906 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1189/2353] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:09 2022] DELETE /identity/v3/projects/b5849686fe0d4b2d8a25c940d2086fa1 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:09.745311 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:09.747318 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:09.747955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:09.786397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:09.789417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:09.894468 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1165/2354] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:50:09.997380 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:09.997884 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:09.998382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.031249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:10.033363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:10.047312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1190/2355] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:10.052085 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.062143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7bbab-f551-40d9-bdf1-4777207c7fa2 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.067291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.067917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.068192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.068457 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.069147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:50:10.069446 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.071076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:10.085488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1166/2356] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:50:10.090781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.110903 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dcee5-1094-44ea-841d-9e79e40c06fa None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.118982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.120113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.120384 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.121077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.122181 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.125527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:10.260054 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1191/2357] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:50:10 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:10.267030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.293614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f1271-cd83-4eb2-a3b3-af30497f21fb admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.376902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:10.400465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.401495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.405203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.410668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.410668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.410668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:10.482784 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1167/2358] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:50:10 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:50:10.489184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.503550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f9df6-cd71-4c8a-95c5-257a09c5f93e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.509771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.510486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.511652 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.512014 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.512737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.514771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:10.532074 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1192/2359] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:50:10.537369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.550636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dfd28-b0a1-493f-987b-1ed3c2ba1a08 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.555402 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d95c5f-3fed-46fa-8a19-9be71521dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.556013 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d95c5f-3fed-46fa-8a19-9be71521dee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.556287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d95c5f-3fed-46fa-8a19-9be71521dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.556580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d95c5f-3fed-46fa-8a19-9be71521dee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.557362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d95c5f-3fed-46fa-8a19-9be71521dee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.559533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d95c5f-3fed-46fa-8a19-9be71521dee6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:10.571913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1168/2360] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:10.578147 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.588941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796674-1aa1-4ef2-89c0-8ac7250d6308 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.594714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ff8c3-8bfb-4ec0-88e1-1029ce0bf83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.595592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff8c3-8bfb-4ec0-88e1-1029ce0bf83a admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.596029 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff8c3-8bfb-4ec0-88e1-1029ce0bf83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.596376 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff8c3-8bfb-4ec0-88e1-1029ce0bf83a admin admin] PATH_INFO: `/v3/projects/dfd1cd86a75c424ca7827b7987206214/users/048d991d051d4013add240411bc182d7/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.619087 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ff8c3-8bfb-4ec0-88e1-1029ce0bf83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd1cd86a75c424ca7827b7987206214, user_id=048d991d051d4013add240411bc182d7, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.622965 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ff8c3-8bfb-4ec0-88e1-1029ce0bf83a admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:10.701035 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1193/2361] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:10 2022] PUT /identity/v3/projects/dfd1cd86a75c424ca7827b7987206214/users/048d991d051d4013add240411bc182d7/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:50:10.720555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.721538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.722114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.761985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] MFA Rules not processed for user `048d991d051d4013add240411bc182d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:10.763480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:10.794268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.794570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.794842 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.817885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] MFA Rules not processed for user `7fa761a568644a9399d825498a51f054`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:10.819154 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:10.878589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1169/2362] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:10.880705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:10.903428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1194/2363] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:50:10.966159 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89209f-55cf-4859-ba6b-7edea0e72f11 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:10.971269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:10.971930 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:10.972178 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:10.972472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:10.973122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:10.975096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.053868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1170/2364] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:50:10 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:11.055558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:11.072047 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282bbe4-32b6-4ca4-9764-98e3fef6dd08 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:11.134597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:11.147323 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:11.149108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.149108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.149108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.149754 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:11.153404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.313762 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:11.315601 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:11.318684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1195/2365] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:50:11 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:11.323525 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:11.336552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012b077-4780-4a14-8c9b-79990fff5bdd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:11.343887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:11.344917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.345467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.345934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.347090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:11.350231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.364431 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1196/2366] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:50:11.369871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:11.371532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:11.383637 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f98e7-57dd-44b2-a601-d58786d639e3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:11.389058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf614b3-472c-43a9-986c-bfb60091636d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:11.389274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:11.389806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.390212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.390484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.391132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:11.392739 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.393041 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fd084-9203-4386-9410-51067f645569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:11.393660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fd084-9203-4386-9410-51067f645569 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.393937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fd084-9203-4386-9410-51067f645569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.394216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fd084-9203-4386-9410-51067f645569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.396047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-488fd084-9203-4386-9410-51067f645569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:11.406509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1197/2367] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:50:11.408030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fd084-9203-4386-9410-51067f645569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:11.409878 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fd084-9203-4386-9410-51067f645569 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.414885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:11.419540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1171/2368] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:11.424836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed99c-85ce-4386-a5cb-6af21090f5aa admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:11.430419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:11.431254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.431894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.432656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] PATH_INFO: `/v3/projects/8b2d229576174a9399bdadb730189561/users/8a5230fbb9984ba0b3303761d3a375b1/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.457571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2d229576174a9399bdadb730189561, user_id=8a5230fbb9984ba0b3303761d3a375b1, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:11.461571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.580584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1198/2369] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:11 2022] PUT /identity/v3/projects/8b2d229576174a9399bdadb730189561/users/8a5230fbb9984ba0b3303761d3a375b1/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:50:11.591295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.591906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.591906 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.628761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] MFA Rules not processed for user `8a5230fbb9984ba0b3303761d3a375b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:11.630132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:11.714060 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1172/2370] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:11.719223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:11.821374 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91242930-988d-427f-86e9-8c3ecbdb910c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:11.828258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:11.828989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:11.833880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:11.833880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:11.833880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:11.833880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:11.961867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1199/2371] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:50:11 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:11.968348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.010631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5e80-eaf6-4cfe-a797-46cc150b3c7c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:12.016841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.018971 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:12.150123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:12.152562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.152562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.152562 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.153244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:12.156005 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9a8dc-b930-4a84-b3cb-fd2b121c97c2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:12.157296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:12.162510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:12.163311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.163729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.164097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.166072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:12.181152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:12.182982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:12.195264 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1200/2372] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:12.286568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1173/2373] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:50:11 2022] POST /identity/v3/users => generated 296 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:12.287665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.307675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d33c5a-802e-4bc9-99b0-f42f0760d8e6 service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:12.317211 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:12.318291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.318776 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.319358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.320665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:12.324493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:12.328795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:12.345958 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1201/2374] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:12.350984 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.371465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33dcbb-0cf1-469d-9ca0-3e22aad8999a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:12.379659 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:12.381043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.381532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.382009 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] PATH_INFO: `/v3/projects/3cc8e09babd5420d8963aa5d30bdbc24/users/e4db8cdfcc1e4572b41342fe6ada121a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.405937 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc8e09babd5420d8963aa5d30bdbc24, user_id=e4db8cdfcc1e4572b41342fe6ada121a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:12.408947 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:12.489852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1174/2375] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:12 2022] PUT /identity/v3/projects/3cc8e09babd5420d8963aa5d30bdbc24/users/e4db8cdfcc1e4572b41342fe6ada121a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:12.494636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.507158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc8c7-a762-4473-8125-f24c0b5a9681 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:12.611558 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f564c6-311e-46cc-a729-997572873f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:12.612591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f564c6-311e-46cc-a729-997572873f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.613111 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f564c6-311e-46cc-a729-997572873f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.613585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f564c6-311e-46cc-a729-997572873f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.614752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f564c6-311e-46cc-a729-997572873f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:12.619176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f564c6-311e-46cc-a729-997572873f41 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:12.641043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1202/2376] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:50:12.645442 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.664758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3cf4-832c-42dc-8bc2-39ba35c2a171 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:12.671281 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:12.671971 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.672329 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.672547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] PATH_INFO: `/v3/projects/3cc8e09babd5420d8963aa5d30bdbc24/users/e4db8cdfcc1e4572b41342fe6ada121a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.674799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc8e09babd5420d8963aa5d30bdbc24, user_id=e4db8cdfcc1e4572b41342fe6ada121a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:12.677665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:12.691772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1175/2377] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:12 2022] PUT /identity/v3/projects/3cc8e09babd5420d8963aa5d30bdbc24/users/e4db8cdfcc1e4572b41342fe6ada121a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:50:12.705128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:12.705639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:12.706195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:12.755389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] MFA Rules not processed for user `e4db8cdfcc1e4572b41342fe6ada121a`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:12.758163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:12.876789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1203/2378] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:12.911640 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:12.915693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:12.966516 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:13.014731 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e35726-08a9-4f90-947b-75becd0a8fae admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:13.021202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:13.022425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:13.022834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:13.023328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:13.041255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:13.044627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:13.063960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1176/2379] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:13.183712 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:13.186142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:13.206207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b146f-5fa0-4dbe-bd5a-6c8efeb70272 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:13.211402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:13.211745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:13.212133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:13.212498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:13.322859 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:13.326685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:13.342615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1204/2380] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:14.758294 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:14.760416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:14.874362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae932ae-a4d7-4a32-8830-16cb7046dc40 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:14.883451 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:14.889519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:14.889519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:14.889519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:14.894044 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:14.924472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:14.928859 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21eb3c-4e5c-4122-93b0-46ef3160e7c1 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:14.961496 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1177/2381] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:18.010019 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:18.057292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=100 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:18.125430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ad8a9-0b95-4aac-91d0-b595b104aafc service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:18.132781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:18.134017 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:18.134451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:18.134967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] PATH_INFO: `/v3/users/048d991d051d4013add240411bc182d7` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:18.137604 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=048d991d051d4013add240411bc182d7)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:18.141542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:18.376192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048d991d051d4013add240411bc182d7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:18.444966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048d991d051d4013add240411bc182d7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:18.463708 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048d991d051d4013add240411bc182d7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:18.465478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048d991d051d4013add240411bc182d7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:18.467023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048d991d051d4013add240411bc182d7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:18.485292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1205/2382] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:18 2022] DELETE /identity/v3/users/048d991d051d4013add240411bc182d7 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:18.490082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:18.490750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:18.490750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:18.513478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:18.514732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:18.599558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1178/2383] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:18.611371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:18.613300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:18.685576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47c7c-fb35-44a2-a4e0-2f2dac401669 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:18.689491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:18.690104 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:18.690385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:18.690658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:18.701843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:18.703221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:18.713964 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1206/2384] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:19.157729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:19.159540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:19.173531 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0083954-dc6b-4831-b0d2-6adef0b677aa None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:19.178692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:19.179346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:19.179628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:19.179899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] PATH_INFO: `/v3/projects/dfd1cd86a75c424ca7827b7987206214` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:19.181158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd1cd86a75c424ca7827b7987206214)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:19.184704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:19.299248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:19.300788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd1cd86a75c424ca7827b7987206214', 'request_id': 'req-187947fa-b9e1-4b24-ac36-be66feab54f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.308308 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd1cd86a75c424ca7827b7987206214', 'request_id': 'req-187947fa-b9e1-4b24-ac36-be66feab54f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.314405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd1cd86a75c424ca7827b7987206214', 'request_id': 'req-187947fa-b9e1-4b24-ac36-be66feab54f1'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.381545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] The token cache is being invalidate because project dfd1cd86a75c424ca7827b7987206214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:19.381841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.382486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.384347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1179/2385] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:19 2022] DELETE /identity/v3/projects/dfd1cd86a75c424ca7827b7987206214 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:19.389099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:19.399113 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907ebc-be1e-4fe1-84a1-07b88d694217 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:19.445468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:19.464864 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:19.465542 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:19.465818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:19.466091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] PATH_INFO: `/v3/users/8a5230fbb9984ba0b3303761d3a375b1` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:19.467294 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5230fbb9984ba0b3303761d3a375b1)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:19.469286 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:19.670295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5230fbb9984ba0b3303761d3a375b1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.749882 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5230fbb9984ba0b3303761d3a375b1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.773118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5230fbb9984ba0b3303761d3a375b1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.773869 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5230fbb9984ba0b3303761d3a375b1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.774503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5230fbb9984ba0b3303761d3a375b1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:19.790309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1207/2386] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:19 2022] DELETE /identity/v3/users/8a5230fbb9984ba0b3303761d3a375b1 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:20.311857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:20.313931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:20.334321 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187947fa-b9e1-4b24-ac36-be66feab54f1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:20.410342 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:20.411056 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:20.411377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:20.411691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] PATH_INFO: `/v3/projects/8b2d229576174a9399bdadb730189561` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:20.413216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2d229576174a9399bdadb730189561)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:20.415480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:20.522490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2d229576174a9399bdadb730189561', 'request_id': 'req-165ed453-78bb-417a-a246-14207a6d6ce3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:20.531088 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2d229576174a9399bdadb730189561', 'request_id': 'req-165ed453-78bb-417a-a246-14207a6d6ce3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:20.536986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2d229576174a9399bdadb730189561', 'request_id': 'req-165ed453-78bb-417a-a246-14207a6d6ce3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:20.607959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] The token cache is being invalidate because project 8b2d229576174a9399bdadb730189561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:20.608272 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:20.608916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:20.610589 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1180/2387] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:20 2022] DELETE /identity/v3/projects/8b2d229576174a9399bdadb730189561 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:20.614851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:20.616463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:20.630326 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd263cb9-3dfe-4b16-9276-067665c4f9de admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:20.699426 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:20.700476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:20.700773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:20.701061 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] PATH_INFO: `/v3/users/e4db8cdfcc1e4572b41342fe6ada121a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:20.702381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4db8cdfcc1e4572b41342fe6ada121a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:20.704381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:20.914227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4db8cdfcc1e4572b41342fe6ada121a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:20.996339 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4db8cdfcc1e4572b41342fe6ada121a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.006012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:21.016371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4db8cdfcc1e4572b41342fe6ada121a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.017284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4db8cdfcc1e4572b41342fe6ada121a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.018023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-10be8ffb-6698-48f2-9132-786070620844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4db8cdfcc1e4572b41342fe6ada121a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.046249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1208/2388] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:20 2022] DELETE /identity/v3/users/e4db8cdfcc1e4572b41342fe6ada121a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:50:21.633022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:21.635526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:21.659017 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ed453-78bb-417a-a246-14207a6d6ce3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:21.738541 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:21.739196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:21.739475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:21.739746 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] PATH_INFO: `/v3/projects/3cc8e09babd5420d8963aa5d30bdbc24` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:21.741104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cc8e09babd5420d8963aa5d30bdbc24)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:21.743146 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:21.754567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:21.848850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cc8e09babd5420d8963aa5d30bdbc24', 'request_id': 'req-3dd57225-d021-4c68-8568-90f482193d51'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.857922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cc8e09babd5420d8963aa5d30bdbc24', 'request_id': 'req-3dd57225-d021-4c68-8568-90f482193d51'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.865473 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cc8e09babd5420d8963aa5d30bdbc24', 'request_id': 'req-3dd57225-d021-4c68-8568-90f482193d51'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.951378 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] The token cache is being invalidate because project 3cc8e09babd5420d8963aa5d30bdbc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:21.951688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.952346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3dd57225-d021-4c68-8568-90f482193d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:21.954202 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1181/2389] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:21 2022] DELETE /identity/v3/projects/3cc8e09babd5420d8963aa5d30bdbc24 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:21.960586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:21.961175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:21.961175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:21.985296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:21.987419 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:22.096174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1209/2390] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:22.105526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.105921 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.106243 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.128091 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:22.129418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:22.139648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1182/2391] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:22.144223 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:22.155033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a76e4-567a-4617-8a03-5fd1d4c6fba9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:22.159880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:22.160503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.160783 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.161085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.161751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:50:22.162053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:22.163685 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:22.178639 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1210/2392] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:50:22.185553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:22.201832 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a0267-96ff-43d4-84b0-5af5bf2c39ac None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:22.209532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:22.210528 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.210993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.211441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.212501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:22.216382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:22.316591 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1183/2393] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:50:22 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:22.322003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:22.341402 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b223e-ca40-4e51-8afe-18c652832a98 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:22.431732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:22.432355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.433216 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.433454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.434397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:22.436033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:22.564886 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1211/2394] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:50:22 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:22.570842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:22.583756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da20202-c979-459a-8177-f4ce5ee1b347 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:22.588574 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:22.589564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.590269 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.590532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.591196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:22.592838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:22.604415 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1184/2395] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:22.611239 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:22.626988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f9b84-b7ad-4de8-bcf5-44e144c7dff9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:22.635092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa413f83-49bf-4cf2-9d5b-bad78810b84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:22.636152 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa413f83-49bf-4cf2-9d5b-bad78810b84c admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.636818 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa413f83-49bf-4cf2-9d5b-bad78810b84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.637332 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa413f83-49bf-4cf2-9d5b-bad78810b84c admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.638509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa413f83-49bf-4cf2-9d5b-bad78810b84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:22.641712 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa413f83-49bf-4cf2-9d5b-bad78810b84c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:22.662576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1212/2396] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:22.666916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:22.677985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ced7d1-02ae-44d1-a64c-948c3f9145f8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:22.683365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:22.684193 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.684578 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.684945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] PATH_INFO: `/v3/projects/7d86436f5a14409e84771aab0415fea2/users/b170a7bae9824abd87491d5ecea89ad3/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.702847 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d86436f5a14409e84771aab0415fea2, user_id=b170a7bae9824abd87491d5ecea89ad3, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:22.705611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:22.813136 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1185/2397] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:22 2022] PUT /identity/v3/projects/7d86436f5a14409e84771aab0415fea2/users/b170a7bae9824abd87491d5ecea89ad3/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:50:22.827009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:22.827210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:22.827473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:22.867520 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] MFA Rules not processed for user `b170a7bae9824abd87491d5ecea89ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:22.869034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:22.914000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:22.955810 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1213/2398] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:22.957301 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.044244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa72046-b56c-4f77-ad65-d45986ec290b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.049493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.055106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.055106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.055106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.055106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.055106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.160823 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1186/2399] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:50:22 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:50:23.162386 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.180373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420f1b6-d9f3-4b06-b774-32874d4be12d None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.208809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.210890 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:23.236040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:23.270701 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.271723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.272270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.272959 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.274023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.278435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.306016 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59864b9-be3b-4da4-be14-876676d67769 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.309621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.310259 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.310539 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.310816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.312754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:23.327133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.335512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.346488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1187/2400] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:50:23.365292 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1214/2401] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:50:23 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:23.371328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.390493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6c30-40ce-4433-80e7-1ebfb4a3b02b service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.398413 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.399501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.399986 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.400479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.402455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.405646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.419752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1188/2402] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:23.424013 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.435921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ebc24-1c87-48c3-82a9-168accfd05dc admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.441247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.442059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.442427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.442779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] PATH_INFO: `/v3/projects/2fd47989b2924a8b86df8b0d0d91b494/users/c11c54bedf4446f49ae3e04c3b33d62a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.461858 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd47989b2924a8b86df8b0d0d91b494, user_id=c11c54bedf4446f49ae3e04c3b33d62a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.466981 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.579922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1215/2403] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:23 2022] PUT /identity/v3/projects/2fd47989b2924a8b86df8b0d0d91b494/users/c11c54bedf4446f49ae3e04c3b33d62a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:23.584401 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.597031 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25accf0f-4c67-442b-b935-6d94222500e8 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.688994 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc918e9-7041-4db8-936f-49ed70c4fd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.689880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc918e9-7041-4db8-936f-49ed70c4fd2f admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.690045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc918e9-7041-4db8-936f-49ed70c4fd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.690214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc918e9-7041-4db8-936f-49ed70c4fd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.690920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc918e9-7041-4db8-936f-49ed70c4fd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.692610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc918e9-7041-4db8-936f-49ed70c4fd2f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.705184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1189/2404] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:23.710647 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.730691 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e8781-6cf7-4e06-b89c-367e77fa3ef6 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:23.739049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:23.740259 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.740737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.741283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] PATH_INFO: `/v3/projects/2fd47989b2924a8b86df8b0d0d91b494/users/c11c54bedf4446f49ae3e04c3b33d62a/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.745894 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd47989b2924a8b86df8b0d0d91b494, user_id=c11c54bedf4446f49ae3e04c3b33d62a, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:23.751400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:23.768548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1216/2405] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:23 2022] PUT /identity/v3/projects/2fd47989b2924a8b86df8b0d0d91b494/users/c11c54bedf4446f49ae3e04c3b33d62a/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:23.781049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:23.781355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:23.781594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:23.816650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] MFA Rules not processed for user `c11c54bedf4446f49ae3e04c3b33d62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:23.818285 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:23.902806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1190/2406] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:50:23.918904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:23.920633 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:24.018277 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ce2ed-1995-47f1-b418-74069dd2f2f9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:24.022089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259be824-858e-4b53-a4cd-03bf53e97b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:24.022703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259be824-858e-4b53-a4cd-03bf53e97b13 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:24.022977 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259be824-858e-4b53-a4cd-03bf53e97b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:24.023247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259be824-858e-4b53-a4cd-03bf53e97b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:24.070512 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:24.088926 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259be824-858e-4b53-a4cd-03bf53e97b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:24.090507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259be824-858e-4b53-a4cd-03bf53e97b13 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:24.100491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1217/2407] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:27.340337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:27.359760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:27.447119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56317bc-5859-4c38-952b-7a553ce50167 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:27.453564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:27.454459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:27.454922 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:27.455353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] PATH_INFO: `/v3/users/b170a7bae9824abd87491d5ecea89ad3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:27.457015 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b170a7bae9824abd87491d5ecea89ad3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:27.460479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:27.654149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b170a7bae9824abd87491d5ecea89ad3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:27.673512 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b170a7bae9824abd87491d5ecea89ad3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:27.749816 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b170a7bae9824abd87491d5ecea89ad3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:27.750671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b170a7bae9824abd87491d5ecea89ad3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:27.751592 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b170a7bae9824abd87491d5ecea89ad3'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:27.778143 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1191/2408] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:27 2022] DELETE /identity/v3/users/b170a7bae9824abd87491d5ecea89ad3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:27.779524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:27.780325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:27.780325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:27.801843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:27.803580 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:27.911075 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1218/2409] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:27.926164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:27.928590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:28.018390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cba088-8be1-483c-a7a0-3656c17bc6d1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:28.023343 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:28.029918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:28.029918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:28.029918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:28.035107 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:28.036568 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:28.047855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1192/2410] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:28.624177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:28.626765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:28.652806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518e10d-4158-4028-b700-990d25fd6a22 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:28.660258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:28.665911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:28.665911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:28.665911 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] PATH_INFO: `/v3/projects/7d86436f5a14409e84771aab0415fea2` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:28.669273 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d86436f5a14409e84771aab0415fea2)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:28.673436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:28.832062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d86436f5a14409e84771aab0415fea2', 'request_id': 'req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:28.840650 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d86436f5a14409e84771aab0415fea2', 'request_id': 'req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:28.910611 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d86436f5a14409e84771aab0415fea2', 'request_id': 'req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:28.924060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] The token cache is being invalidate because project 7d86436f5a14409e84771aab0415fea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:28.924379 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:28.925322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:28.927443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1219/2411] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:28 2022] DELETE /identity/v3/projects/7d86436f5a14409e84771aab0415fea2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:28.931315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:28.942183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aae00-0d4b-4f32-806a-9063e845c997 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:28.979049 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:29.007792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:29.008534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:29.008818 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:29.009162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] PATH_INFO: `/v3/users/c11c54bedf4446f49ae3e04c3b33d62a` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:29.010405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11c54bedf4446f49ae3e04c3b33d62a)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:29.014076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:29.213520 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11c54bedf4446f49ae3e04c3b33d62a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:29.230040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11c54bedf4446f49ae3e04c3b33d62a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:29.320792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11c54bedf4446f49ae3e04c3b33d62a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:29.321642 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11c54bedf4446f49ae3e04c3b33d62a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:29.322347 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d1fa9ef0-73bd-4450-a3d6-85521fe7155b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11c54bedf4446f49ae3e04c3b33d62a'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:29.337587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1193/2412] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:28 2022] DELETE /identity/v3/users/c11c54bedf4446f49ae3e04c3b33d62a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:50:29.729705 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:29.731398 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:29.752799 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c324c3-ad0b-4d8e-a2c6-d467a3cc6168 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:29.829620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:29.830570 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:29.830995 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:29.831385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] PATH_INFO: `/v3/projects/2fd47989b2924a8b86df8b0d0d91b494` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:29.832684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd47989b2924a8b86df8b0d0d91b494)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:29.835235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:29.843530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:29.991053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd47989b2924a8b86df8b0d0d91b494', 'request_id': 'req-fa510100-6cdd-46cd-88d1-f4c012d32b1c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:29.999549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd47989b2924a8b86df8b0d0d91b494', 'request_id': 'req-fa510100-6cdd-46cd-88d1-f4c012d32b1c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:30.063877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd47989b2924a8b86df8b0d0d91b494', 'request_id': 'req-fa510100-6cdd-46cd-88d1-f4c012d32b1c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:30.079972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] The token cache is being invalidate because project 2fd47989b2924a8b86df8b0d0d91b494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:30.080348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:30.081487 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-fa510100-6cdd-46cd-88d1-f4c012d32b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:30.084092 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1220/2413] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:29 2022] DELETE /identity/v3/projects/2fd47989b2924a8b86df8b0d0d91b494 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:30.089869 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.090390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.090494 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.114418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:30.115647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:30.179226 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1194/2414] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:30.188037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.188320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.188595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.211825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:30.213244 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:30.222717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1221/2415] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:30.227320 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.238738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4a4b3-afd0-45ec-8005-af60f0d8798d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.244169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:30.244810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.245925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.246629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.246926 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:50:30.247232 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:30.248894 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:30.263936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1195/2416] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:50:30.270175 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.281418 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda47f6-10ee-46a0-85f4-c80cbc0a8f18 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.287407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:30.288267 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.288597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.288902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.289710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:30.292975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:30.387194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1222/2417] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:50:30 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:50:30.391992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.403186 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b32d2b-42d3-4432-80db-6952757b2af4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.454950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:30.485090 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:30.486184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.486973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.487479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.488533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:30.491815 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:30.580408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1196/2418] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:50:30 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:50:30.586948 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.598587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646413-ec55-4af6-9077-8ff0646a3abf admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.603605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:30.604241 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.604636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.604910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.605613 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:30.607263 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:30.619578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1223/2419] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:30.625913 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.638877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341611d-6247-4de0-875d-9aa39a460e74 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.644691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04776a51-84f5-4790-a896-7700f9c3dd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:30.645417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04776a51-84f5-4790-a896-7700f9c3dd30 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.645697 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04776a51-84f5-4790-a896-7700f9c3dd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.645967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04776a51-84f5-4790-a896-7700f9c3dd30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.646618 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04776a51-84f5-4790-a896-7700f9c3dd30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:30.648284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04776a51-84f5-4790-a896-7700f9c3dd30 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:30.660888 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1197/2420] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:50:30.665951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.678369 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95ef6e-401c-4bf4-9914-420ad14b5f7c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.686457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:30.687185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.687467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.687741 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] PATH_INFO: `/v3/projects/2e834030bdbc4a52927d049a6c467aff/users/dc8578f749604dfe9e476c4431070761/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.706318 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e834030bdbc4a52927d049a6c467aff, user_id=dc8578f749604dfe9e476c4431070761, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:30.709201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:30.790752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1224/2421] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:50:30 2022] PUT /identity/v3/projects/2e834030bdbc4a52927d049a6c467aff/users/dc8578f749604dfe9e476c4431070761/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:30.803237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:30.803761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:30.804229 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:30.842691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] MFA Rules not processed for user `dc8578f749604dfe9e476c4431070761`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:30.844626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:30.918635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1198/2422] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:30.930142 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:30.931922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:30.995782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858261a-3a05-4f91-9662-789d0ff68df7 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:30.999694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8f29e-ff83-4c70-ad59-88b1b7b19c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:31.000312 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f29e-ff83-4c70-ad59-88b1b7b19c7b service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:31.000586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f29e-ff83-4c70-ad59-88b1b7b19c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:31.000850 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f29e-ff83-4c70-ad59-88b1b7b19c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:31.010070 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8f29e-ff83-4c70-ad59-88b1b7b19c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:31.011539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8f29e-ff83-4c70-ad59-88b1b7b19c7b service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:31.022425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1225/2423] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:40.914292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:41.016806 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:50:55.313573 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:55.381067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625aaee8-efc9-4b11-be88-04f2ec721f72 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:55.385674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:55.386830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:55.387104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:55.387372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] PATH_INFO: `/v3/users/dc8578f749604dfe9e476c4431070761` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:55.388506 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8578f749604dfe9e476c4431070761)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:55.390479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:55.575177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8578f749604dfe9e476c4431070761'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:55.587227 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8578f749604dfe9e476c4431070761'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:55.663564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8578f749604dfe9e476c4431070761'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:55.664199 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8578f749604dfe9e476c4431070761'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:55.664801 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8578f749604dfe9e476c4431070761'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:55.678558 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1199/2424] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:50:55 2022] DELETE /identity/v3/users/dc8578f749604dfe9e476c4431070761 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:50:55.683817 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:55.684346 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:55.684461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:55.703210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:50:55.704394 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:55.763636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1226/2425] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:50:55.773444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:55.774950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:55.832772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afe9ba-4169-4893-a124-97f5c1e7c3c6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:55.836383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:55.836981 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:55.837288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:55.837552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:55.847350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:55.848777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:55.859008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1200/2426] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:50:56.291543 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:50:56.292947 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:50:56.305012 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0748a-d0b8-4218-876b-868b6b583081 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:50:56.310462 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:50:56.311840 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:50:56.312296 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:50:56.312578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] PATH_INFO: `/v3/projects/2e834030bdbc4a52927d049a6c467aff` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:50:56.313812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e834030bdbc4a52927d049a6c467aff)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:50:56.316399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:50:56.436746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e834030bdbc4a52927d049a6c467aff', 'request_id': 'req-290291b7-b093-4dea-a47a-90e175173a3d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:56.443272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e834030bdbc4a52927d049a6c467aff', 'request_id': 'req-290291b7-b093-4dea-a47a-90e175173a3d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:56.514507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e834030bdbc4a52927d049a6c467aff', 'request_id': 'req-290291b7-b093-4dea-a47a-90e175173a3d'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:56.525541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] The token cache is being invalidate because project 2e834030bdbc4a52927d049a6c467aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:50:56.525831 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:56.526452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:50:56.527934 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1227/2427] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:50:56 2022] DELETE /identity/v3/projects/2e834030bdbc4a52927d049a6c467aff => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:05.854627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:06.522284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:29.113069 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:29.122024 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50470ab-6be7-4de0-ad12-00156fd94f24 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:29.171663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:29.172309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:29.172588 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:29.172857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:29.174674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:29.253251 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:29.254695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:29.264901 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1201/2428] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:30.198673 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:30.200719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:30.273221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290291b7-b093-4dea-a47a-90e175173a3d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:30.278372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:30.278992 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:30.279271 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:30.279541 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:30.281331 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:30.294009 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:30.295595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:30.306724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1228/2429] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:30.355567 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:30.357112 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:30.370263 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8d6d6-be37-4201-8928-a2040f72a2b4 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:30.375257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:30.375895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:30.376170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:30.376450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] PATH_INFO: `/v3/users/7fa761a568644a9399d825498a51f054` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:30.377684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa761a568644a9399d825498a51f054)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:30.381114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:30.592038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa761a568644a9399d825498a51f054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:30.607302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa761a568644a9399d825498a51f054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:30.708774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa761a568644a9399d825498a51f054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:30.709416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa761a568644a9399d825498a51f054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:30.710020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-1158cdfe-a69d-4f22-825d-bb7c0488515e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa761a568644a9399d825498a51f054'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:30.724141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1202/2430] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:30 2022] DELETE /identity/v3/users/7fa761a568644a9399d825498a51f054 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:31.006829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.019972 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63187ffe-46ee-41ff-a93e-6f93f33d4fc1 service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.080086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.080755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.084834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.084834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] PATH_INFO: `/v3/projects/e428f318654f446db48445e4570ae2d3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.084834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e428f318654f446db48445e4570ae2d3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.084834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.192829 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e428f318654f446db48445e4570ae2d3', 'request_id': 'req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:31.200026 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e428f318654f446db48445e4570ae2d3', 'request_id': 'req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:31.279422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e428f318654f446db48445e4570ae2d3', 'request_id': 'req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:31.291126 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] The token cache is being invalidate because project e428f318654f446db48445e4570ae2d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:31.291473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:31.292169 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7043d2ab-e85e-4f1d-9b92-f427e25f4d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:31.293933 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1229/2431] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:31 2022] DELETE /identity/v3/projects/e428f318654f446db48445e4570ae2d3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:31.298710 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.298996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.299276 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.319662 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:31.320858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:31.338475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:31.382144 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1203/2432] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:31.390901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.391186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.391458 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.411938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:31.413173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:31.421120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1230/2433] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:31.425355 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.436856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c8ec-5c65-4a58-8239-f9ae63e7c44d None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.441475 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.447214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.447214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.447214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.447214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:31.447214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.447214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.457799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1204/2434] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:31.462358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.472067 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70782e-4921-4e80-bb45-65bdb80dfaa1 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.476717 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.477344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.477620 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.477887 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.478568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.480631 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.491828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:31.560473 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1231/2435] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:31 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:31.564668 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.575369 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737abd17-93b5-440b-8ac2-3a9e8157fb12 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.632575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.633223 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.633488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.633760 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.634421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.636669 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.736629 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1205/2436] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:31 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:31.740980 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.752910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122656fb-f178-428c-a38f-adfdf1f9748f admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.757608 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.758210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.758489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.758763 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.759403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.761781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.771811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1232/2437] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:31.776150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.786319 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20afefc-688e-45a9-a2da-ed3cef479979 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.795400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb2ba-bad8-4854-a17d-bec73e041fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.796018 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb2ba-bad8-4854-a17d-bec73e041fde admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.796302 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb2ba-bad8-4854-a17d-bec73e041fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.796580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb2ba-bad8-4854-a17d-bec73e041fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.797244 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb2ba-bad8-4854-a17d-bec73e041fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.798852 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb2ba-bad8-4854-a17d-bec73e041fde admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.811293 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1206/2438] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:31.817627 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:31.829577 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef368e-0b18-4fd9-aceb-344c36e9669b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:31.834193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:31.834842 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.835124 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.835401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] PATH_INFO: `/v3/projects/9dfe7b7c29f342faa0862767eac362dd/users/46cc19d4447d4440a18a716e6f1662fa/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.851428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfe7b7c29f342faa0862767eac362dd, user_id=46cc19d4447d4440a18a716e6f1662fa, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:31.854748 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:31.947039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1233/2439] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:31 2022] PUT /identity/v3/projects/9dfe7b7c29f342faa0862767eac362dd/users/46cc19d4447d4440a18a716e6f1662fa/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:31.958291 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:31.958570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:31.958872 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:31.987792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] MFA Rules not processed for user `46cc19d4447d4440a18a716e6f1662fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:31.988897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:32.053530 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1207/2440] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:32.067583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:32.069428 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:32.144230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7aeb-30d4-419e-8ad2-8ade9e8b59af admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:32.149796 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39a365-2bda-434e-822c-39f247f0a2c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:32.151006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39a365-2bda-434e-822c-39f247f0a2c1 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:32.151235 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39a365-2bda-434e-822c-39f247f0a2c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:32.151500 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39a365-2bda-434e-822c-39f247f0a2c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:32.159482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39a365-2bda-434e-822c-39f247f0a2c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:32.161072 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39a365-2bda-434e-822c-39f247f0a2c1 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:32.170649 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1234/2441] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:32.385970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:32.405820 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:32.450709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad7164-dc0c-4e50-a47d-ce5d928ff304 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:32.455253 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:32.455875 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:32.456150 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:32.456421 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] PATH_INFO: `/v3/users/46cc19d4447d4440a18a716e6f1662fa` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:32.457584 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cc19d4447d4440a18a716e6f1662fa)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:32.459532 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:32.631180 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cc19d4447d4440a18a716e6f1662fa'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:32.643749 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cc19d4447d4440a18a716e6f1662fa'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:32.701020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cc19d4447d4440a18a716e6f1662fa'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:32.701653 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cc19d4447d4440a18a716e6f1662fa'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:32.702249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cc19d4447d4440a18a716e6f1662fa'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:32.715510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1208/2442] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:32 2022] DELETE /identity/v3/users/46cc19d4447d4440a18a716e6f1662fa => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:32.720255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:32.720743 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:32.720843 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:32.739839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:32.741046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:32.799391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1235/2443] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:32.809961 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:32.811352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:32.870316 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddfe49-0197-49eb-8892-0a6558883489 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:32.874248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09deb6-7ce4-4a26-a06a-393de4c920d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:32.874855 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09deb6-7ce4-4a26-a06a-393de4c920d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:32.876153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09deb6-7ce4-4a26-a06a-393de4c920d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:32.876153 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09deb6-7ce4-4a26-a06a-393de4c920d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:32.886099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09deb6-7ce4-4a26-a06a-393de4c920d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:32.887617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09deb6-7ce4-4a26-a06a-393de4c920d3 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:32.897796 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1209/2444] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:33.270297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:33.271868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:33.285046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209b929-b1df-442b-95d1-fbaca5770da4 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:33.289827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:33.290468 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.291272 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.291545 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] PATH_INFO: `/v3/projects/9dfe7b7c29f342faa0862767eac362dd` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.293122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dfe7b7c29f342faa0862767eac362dd)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:33.295115 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:33.375885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:33.384716 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dfe7b7c29f342faa0862767eac362dd', 'request_id': 'req-ff591c9c-df31-449b-b074-56a8970caebb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:33.391189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dfe7b7c29f342faa0862767eac362dd', 'request_id': 'req-ff591c9c-df31-449b-b074-56a8970caebb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:33.485484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dfe7b7c29f342faa0862767eac362dd', 'request_id': 'req-ff591c9c-df31-449b-b074-56a8970caebb'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:33.498493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] The token cache is being invalidate because project 9dfe7b7c29f342faa0862767eac362dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:33.498786 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:33.499464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-ff591c9c-df31-449b-b074-56a8970caebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:33.501039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1236/2445] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:33 2022] DELETE /identity/v3/projects/9dfe7b7c29f342faa0862767eac362dd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:33.506145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.506706 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.506830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.529370 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:33.530671 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:33.568062 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:33.596490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1210/2446] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:33.605683 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.605969 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.606247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.627186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:33.628422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:33.636744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1237/2447] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:33.641128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:33.653743 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8edd-ad37-4a2e-932e-7c40a2fd3ae8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:33.658514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:33.659117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.659492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.659708 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.660353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:33.660650 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:33.662295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:33.675349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1211/2448] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:33.679891 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:33.689773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af205-70f6-4548-aa98-1c845178cc72 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:33.694524 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:33.695132 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.695407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.695674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.696300 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:33.698234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:33.796553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1238/2449] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:33 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:33.800537 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:33.810783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c917dac-3040-4a73-81a4-a012d68434b4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:33.867973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:33.868580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.868879 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.869247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.870052 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:33.871951 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:33.971699 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1212/2450] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:33 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:51:33.976094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:33.986587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f78b9-056b-4e38-b998-6959335350b4 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:33.991358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:33.991945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:33.992217 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:33.992483 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:33.993150 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:33.994996 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.005024 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1239/2451] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:34.008908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.019551 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbc902-1cc1-4af5-96b9-baa69bbbd789 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.024201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30147b31-7ec7-4111-8a71-42299f1d0839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.024789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147b31-7ec7-4111-8a71-42299f1d0839 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.025164 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147b31-7ec7-4111-8a71-42299f1d0839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.025991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147b31-7ec7-4111-8a71-42299f1d0839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.026655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30147b31-7ec7-4111-8a71-42299f1d0839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.028245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30147b31-7ec7-4111-8a71-42299f1d0839 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.038774 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1213/2452] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:34.043400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.053601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630a10-8ce8-4725-b6b5-cb3c6f2667fd admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.058252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.058898 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.059176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.059446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] PATH_INFO: `/v3/projects/db94bb82d7bd4572a21c982d736d3d9b/users/851ef82a547048159394aab37984a668/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.074460 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db94bb82d7bd4572a21c982d736d3d9b, user_id=851ef82a547048159394aab37984a668, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.077135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.154345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1240/2453] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:34 2022] PUT /identity/v3/projects/db94bb82d7bd4572a21c982d736d3d9b/users/851ef82a547048159394aab37984a668/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:34.164738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.165047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.168360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.193987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] MFA Rules not processed for user `851ef82a547048159394aab37984a668`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:34.197284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:34.260884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1214/2454] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:34.266210 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.327779 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751147b2-1ebe-4059-a1cb-a66b6b800b4e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.332653 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.333270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.333552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.333822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.334493 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.336385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.436411 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1241/2455] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:34 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:34.440571 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.451982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952624-1677-4ecc-aa5a-9f6240cdb287 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.486073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:34.510886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.511514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.511758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.512023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.512636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.514635 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.582992 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1215/2456] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:34 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:34.586880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.598018 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c530-1239-4d92-84f6-2bfa2b8a3ff2 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.602921 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.604182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.604464 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.604772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.605470 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.607106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.618077 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1242/2457] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:34.622685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.632764 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa777a-e084-437e-a900-b9345a5a7348 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.638212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.638856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.639131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.639399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] PATH_INFO: `/v3/projects/53af187a6a2a468b97e1968702288ad3/users/c6e284fd26714c90ba473e6973ffb75f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.655132 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=53af187a6a2a468b97e1968702288ad3, user_id=c6e284fd26714c90ba473e6973ffb75f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.657943 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.749063 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1216/2458] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:34 2022] PUT /identity/v3/projects/53af187a6a2a468b97e1968702288ad3/users/c6e284fd26714c90ba473e6973ffb75f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:34.753248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.763846 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac4d4-e646-4fb7-b52c-452d219d8ab7 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.796171 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:34.819436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21631b6-dd3a-48d8-857c-7287c52ad409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.820066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21631b6-dd3a-48d8-857c-7287c52ad409 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.820289 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21631b6-dd3a-48d8-857c-7287c52ad409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.820606 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21631b6-dd3a-48d8-857c-7287c52ad409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.821269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21631b6-dd3a-48d8-857c-7287c52ad409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.823329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21631b6-dd3a-48d8-857c-7287c52ad409 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.836144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1243/2459] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:34.837351 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.848203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901fa7f-ab1d-412f-87d2-ccf68efe20bd admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:34.852840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:34.853621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.853897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.854161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] PATH_INFO: `/v3/projects/53af187a6a2a468b97e1968702288ad3/users/c6e284fd26714c90ba473e6973ffb75f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.856053 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53af187a6a2a468b97e1968702288ad3, user_id=c6e284fd26714c90ba473e6973ffb75f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:34.858739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:34.868904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1217/2460] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:34 2022] PUT /identity/v3/projects/53af187a6a2a468b97e1968702288ad3/users/c6e284fd26714c90ba473e6973ffb75f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:34.880773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:34.881046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:34.881319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:34.913367 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] MFA Rules not processed for user `c6e284fd26714c90ba473e6973ffb75f`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:34.914565 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:34.983885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1244/2461] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:34.995908 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:34.997620 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:35.055693 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c15b99-76f3-4a42-aa26-ca8404b22280 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:35.059987 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d1612-b99f-4fe9-a139-568e90d4f02f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:35.060590 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d1612-b99f-4fe9-a139-568e90d4f02f service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:35.060868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d1612-b99f-4fe9-a139-568e90d4f02f service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:35.061167 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d1612-b99f-4fe9-a139-568e90d4f02f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:35.070437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d1612-b99f-4fe9-a139-568e90d4f02f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:35.071856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d1612-b99f-4fe9-a139-568e90d4f02f service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:35.082304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1218/2462] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:35.112185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:35.174397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5512a-0554-443f-be57-adb912b696b9 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:35.178978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:35.185827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:35.185827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:35.185827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] PATH_INFO: `/v3/users/851ef82a547048159394aab37984a668` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:35.185827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=851ef82a547048159394aab37984a668)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:35.185827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:35.345505 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851ef82a547048159394aab37984a668'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:35.445305 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851ef82a547048159394aab37984a668'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:35.450656 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:35.459040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851ef82a547048159394aab37984a668'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:35.461692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851ef82a547048159394aab37984a668'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:35.461692 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851ef82a547048159394aab37984a668'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:35.472970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1245/2463] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:35 2022] DELETE /identity/v3/users/851ef82a547048159394aab37984a668 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:35.477521 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:35.477802 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:35.478072 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:35.498287 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:35.499480 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:35.559800 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1219/2464] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:35.570107 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:35.571837 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:35.632283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aab567-4ac4-45d0-a6c7-8797fc2cd137 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:35.636674 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:35.637307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:35.637592 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:35.637851 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:35.647066 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:35.648472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:35.658945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1246/2465] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:36.068495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:36.070210 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:36.083988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052b11b-08e7-4ea4-a3bb-f97a067f16ad None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:36.088934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:36.089600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:36.089870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:36.090141 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] PATH_INFO: `/v3/projects/db94bb82d7bd4572a21c982d736d3d9b` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:36.092448 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db94bb82d7bd4572a21c982d736d3d9b)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:36.094683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:36.205507 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:36.230432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db94bb82d7bd4572a21c982d736d3d9b', 'request_id': 'req-0f670877-9766-4d49-9411-d7ec4cf01f67'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.241215 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db94bb82d7bd4572a21c982d736d3d9b', 'request_id': 'req-0f670877-9766-4d49-9411-d7ec4cf01f67'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.248522 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db94bb82d7bd4572a21c982d736d3d9b', 'request_id': 'req-0f670877-9766-4d49-9411-d7ec4cf01f67'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.348233 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] The token cache is being invalidate because project db94bb82d7bd4572a21c982d736d3d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:36.348518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.349184 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.350667 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1220/2466] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:36 2022] DELETE /identity/v3/projects/db94bb82d7bd4572a21c982d736d3d9b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:36.355974 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:36.366973 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148205c-5a5d-44ad-8384-0ded57bae711 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:36.433616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:36.434252 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:36.434530 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:36.434803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] PATH_INFO: `/v3/users/c6e284fd26714c90ba473e6973ffb75f` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:36.435950 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e284fd26714c90ba473e6973ffb75f)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:36.437912 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:36.654618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e284fd26714c90ba473e6973ffb75f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.742406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e284fd26714c90ba473e6973ffb75f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.755626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e284fd26714c90ba473e6973ffb75f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.757853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e284fd26714c90ba473e6973ffb75f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.757853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b161b5f-d1e7-49b2-98aa-453f922d8024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e284fd26714c90ba473e6973ffb75f'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:36.765135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:36.770176 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1247/2467] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:36 2022] DELETE /identity/v3/users/c6e284fd26714c90ba473e6973ffb75f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:37.196874 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:37.199466 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:37.225383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f670877-9766-4d49-9411-d7ec4cf01f67 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:37.297066 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:37.297809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.298084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.298352 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] PATH_INFO: `/v3/projects/53af187a6a2a468b97e1968702288ad3` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.299523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53af187a6a2a468b97e1968702288ad3)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:37.301738 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:37.409360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53af187a6a2a468b97e1968702288ad3', 'request_id': 'req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:37.416576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53af187a6a2a468b97e1968702288ad3', 'request_id': 'req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:37.423082 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53af187a6a2a468b97e1968702288ad3', 'request_id': 'req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:37.485536 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] The token cache is being invalidate because project 53af187a6a2a468b97e1968702288ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:37.485822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:37.486458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9df1fb8b-9069-40b1-b9a4-0e1b891f5380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:37.488388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1221/2468] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:37 2022] DELETE /identity/v3/projects/53af187a6a2a468b97e1968702288ad3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:37.494615 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.495189 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.495401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.516726 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:37.517978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:37.588540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1248/2469] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:51:37.590219 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.590535 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.590768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.611366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:37.612652 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:37.621877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1222/2470] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:51:37.626329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:37.636325 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e3c2-56c9-4f58-b5eb-8cdc70a5e9aa None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:37.640910 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:37.641539 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.641814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.642079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.642719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:37.643015 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:37.644597 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:37.656642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1249/2471] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:37.660702 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:37.670983 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cda25-bb6a-4931-b870-9088ea61dda7 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:37.675829 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:37.676470 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.676747 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.677045 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.677690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:37.679587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:37.800907 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1223/2472] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:37 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:37.805042 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:37.815749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b93ae9-a630-4ae0-8871-e5f787bda48b admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:37.873085 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:37.873689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.873967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.874232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.874863 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:37.876822 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:37.949099 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1250/2473] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:37 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:37.953300 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:37.964279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb6eb8-58d0-4e4a-acc0-117af28d2002 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:37.968929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:37.969604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:37.969880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:37.970151 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:37.970790 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:37.972395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:37.983288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1224/2474] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:37.992605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.003056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c682e7-2f12-46c7-859c-27cccf0dc8e1 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.007609 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262421-a0f3-4fcf-9304-fa1c9a89dff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.008199 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262421-a0f3-4fcf-9304-fa1c9a89dff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.008474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262421-a0f3-4fcf-9304-fa1c9a89dff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.008744 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262421-a0f3-4fcf-9304-fa1c9a89dff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.009465 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262421-a0f3-4fcf-9304-fa1c9a89dff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.011049 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262421-a0f3-4fcf-9304-fa1c9a89dff7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.021767 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1251/2475] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:38.026770 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.033242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:38.036927 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5791-52f1-44b1-874b-146f41e271da admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.042191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.042842 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.043169 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.043471 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] PATH_INFO: `/v3/projects/f9726e8bffc9435fab3ea06e52e28763/users/3580ed5290e9438c8af036fdae3eb739/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.059118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9726e8bffc9435fab3ea06e52e28763, user_id=3580ed5290e9438c8af036fdae3eb739, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.061974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.158858 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1225/2476] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:38 2022] PUT /identity/v3/projects/f9726e8bffc9435fab3ea06e52e28763/users/3580ed5290e9438c8af036fdae3eb739/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:38.172703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.172982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.173310 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.202400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] MFA Rules not processed for user `3580ed5290e9438c8af036fdae3eb739`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:38.203595 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:38.258962 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:38.269764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1252/2477] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:38.274280 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.341756 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91abfb8-f333-4265-9878-1fdc31d2fe3c admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.346504 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.347109 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.347390 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.347666 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.348304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.350274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.440955 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1226/2478] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:38 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:38.444872 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.455659 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afa407-dbfb-492c-866a-903079270bb5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.515190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.515915 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.516044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.516247 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.516885 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.518861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.598001 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1253/2479] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:38 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:51:38.601957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.613600 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faecc9-c54b-473b-afca-e51d79b31a5e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.618398 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.620414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.623819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.623819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.623819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.623819 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.633991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1227/2480] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:38.638158 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.648118 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72510c6c-d27c-4935-8138-4addc62a72fc admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.653283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.653937 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.654215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.654485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] PATH_INFO: `/v3/projects/8007d89ecd5d406995a16df4fc1b2c91/users/f7d46fc3b47842e8afc001a7b4168a6f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.670101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8007d89ecd5d406995a16df4fc1b2c91, user_id=f7d46fc3b47842e8afc001a7b4168a6f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.672725 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.741106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1254/2481] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:38 2022] PUT /identity/v3/projects/8007d89ecd5d406995a16df4fc1b2c91/users/f7d46fc3b47842e8afc001a7b4168a6f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:38.745649 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.755696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a94f8b-8293-4f8d-a0ea-5106bc424aa1 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.813709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca635e98-e096-474d-b93a-b4386336fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.814237 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca635e98-e096-474d-b93a-b4386336fbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.814517 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca635e98-e096-474d-b93a-b4386336fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.814791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca635e98-e096-474d-b93a-b4386336fbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.815428 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca635e98-e096-474d-b93a-b4386336fbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.817058 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca635e98-e096-474d-b93a-b4386336fbf2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.827230 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1228/2482] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:38.831354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:38.842523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585bd7-f2e0-42b3-ab3c-a19d8f3b5013 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:38.847090 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:38.847736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.848011 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.848278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] PATH_INFO: `/v3/projects/8007d89ecd5d406995a16df4fc1b2c91/users/f7d46fc3b47842e8afc001a7b4168a6f/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.850517 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8007d89ecd5d406995a16df4fc1b2c91, user_id=f7d46fc3b47842e8afc001a7b4168a6f, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:38.853999 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:38.864626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1255/2483] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:38 2022] PUT /identity/v3/projects/8007d89ecd5d406995a16df4fc1b2c91/users/f7d46fc3b47842e8afc001a7b4168a6f/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:38.931996 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:38.932328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:38.932603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:38.962198 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] MFA Rules not processed for user `f7d46fc3b47842e8afc001a7b4168a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:38.963723 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:39.004576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:39.032891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1229/2484] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:39.046288 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:39.049857 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:39.105536 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ad86-7833-4412-b791-bc059765d8a5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:39.109440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:39.110046 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:39.110321 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:39.110587 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:39.120618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:39.122060 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:39.131495 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1256/2485] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:40.197627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:40.207371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc563f-9c55-4ad1-b289-93b7d29fbe6f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:40.212098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7d46fc3b47842e8afc001a7b4168a6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8007d89ecd5d406995a16df4fc1b2c91', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:40.213458 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:40.216734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:40.216734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:40.216734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:40.216734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:40.216734 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:40.228974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1230/2486] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:40.233559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:40.243283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c70b-a880-45e5-a009-3c73ee8f7a1f service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:40.248825 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7d46fc3b47842e8afc001a7b4168a6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8007d89ecd5d406995a16df4fc1b2c91', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:40.249482 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:40.249759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:40.250030 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:40.250651 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:40.252523 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:40.337938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1257/2487] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:40 2022] POST /identity/v3/projects => generated 354 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:40.342405 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:40.352689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94189717-eabe-4274-8195-9f25769ebccd tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:40.413576 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7d46fc3b47842e8afc001a7b4168a6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8007d89ecd5d406995a16df4fc1b2c91', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:40.414101 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:40.414381 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:40.414655 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:40.415305 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:40.415674 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:40.417304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:40.429322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1231/2488] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:40.469699 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:40.480834 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea99c6-a00d-44ad-873e-f288e46f2f6a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:40.485897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7d46fc3b47842e8afc001a7b4168a6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8007d89ecd5d406995a16df4fc1b2c91', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:40.486554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:40.486835 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:40.487109 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] PATH_INFO: `/v3/projects/c621929ebcf048fd825d88300dae391f` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:40.488293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c621929ebcf048fd825d88300dae391f)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:40.490344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:40.517645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:40.607804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c621929ebcf048fd825d88300dae391f', 'request_id': 'req-c5a48ead-3be4-406c-9249-277efd71d69a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:40.615108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c621929ebcf048fd825d88300dae391f', 'request_id': 'req-c5a48ead-3be4-406c-9249-277efd71d69a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:40.695023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c621929ebcf048fd825d88300dae391f', 'request_id': 'req-c5a48ead-3be4-406c-9249-277efd71d69a'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:40.706738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] The token cache is being invalidate because project c621929ebcf048fd825d88300dae391f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:40.707023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:40.707721 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-c5a48ead-3be4-406c-9249-277efd71d69a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:40.709349 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1258/2489] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:40 2022] DELETE /identity/v3/projects/c621929ebcf048fd825d88300dae391f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:51:41.332807 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:41.334647 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:41.400648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2aed3-afe7-4441-8439-25207ae5908a tempest-VolumeQuotaClassesTest-1862018645 tempest-VolumeQuotaClassesTest-1862018645-project-admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:41.456362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:41.457026 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:41.457309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:41.457579 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] PATH_INFO: `/v3/users/3580ed5290e9438c8af036fdae3eb739` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:41.458720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3580ed5290e9438c8af036fdae3eb739)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:41.460628 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:41.465553 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=100 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:41.666425 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3580ed5290e9438c8af036fdae3eb739'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:41.678717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3580ed5290e9438c8af036fdae3eb739'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:41.748023 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3580ed5290e9438c8af036fdae3eb739'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:41.748631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3580ed5290e9438c8af036fdae3eb739'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:41.749286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3580ed5290e9438c8af036fdae3eb739'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:41.762133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1232/2490] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:41 2022] DELETE /identity/v3/users/3580ed5290e9438c8af036fdae3eb739 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:41.766672 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:41.767157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:41.767297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:41.786719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:41.787982 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:41.847486 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1259/2491] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:41.857988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:41.859372 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:41.919433 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e655ad-b233-4dd7-b4a1-b06af82f287d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:41.922929 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:41.923534 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:41.923810 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:41.924080 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:41.933257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:41.934690 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:41.944543 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1233/2492] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:42.327086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:42.327883 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:42.342050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd5222-64de-4490-9755-7e77e97bdc54 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:42.347051 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:42.347688 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:42.347960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:42.348231 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] PATH_INFO: `/v3/projects/f9726e8bffc9435fab3ea06e52e28763` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:42.349372 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9726e8bffc9435fab3ea06e52e28763)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:42.351278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:42.458094 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9726e8bffc9435fab3ea06e52e28763', 'request_id': 'req-d6ada67e-2352-4272-9c18-f088b862a780'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.466448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9726e8bffc9435fab3ea06e52e28763', 'request_id': 'req-d6ada67e-2352-4272-9c18-f088b862a780'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.540693 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9726e8bffc9435fab3ea06e52e28763', 'request_id': 'req-d6ada67e-2352-4272-9c18-f088b862a780'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.552050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] The token cache is being invalidate because project f9726e8bffc9435fab3ea06e52e28763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:42.552335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.552960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.554456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1260/2493] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:42 2022] DELETE /identity/v3/projects/f9726e8bffc9435fab3ea06e52e28763 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:42.559188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:42.569042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffb25a-3805-4853-9524-92e5886f8bea service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:42.626204 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:42.626833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:42.627128 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:42.627376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] PATH_INFO: `/v3/users/f7d46fc3b47842e8afc001a7b4168a6f` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:42.628495 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d46fc3b47842e8afc001a7b4168a6f)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:42.630447 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:42.716416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:42.825751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d46fc3b47842e8afc001a7b4168a6f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.838932 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d46fc3b47842e8afc001a7b4168a6f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.929782 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d46fc3b47842e8afc001a7b4168a6f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.930744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d46fc3b47842e8afc001a7b4168a6f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.932054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-afda0af1-338e-46af-9844-f978a7b47fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d46fc3b47842e8afc001a7b4168a6f'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:42.945303 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1234/2494] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:42 2022] DELETE /identity/v3/users/f7d46fc3b47842e8afc001a7b4168a6f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:43.349448 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:43.351025 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:43.365443 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ada67e-2352-4272-9c18-f088b862a780 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:43.406678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:43.435063 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:43.435698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:43.435975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:43.436253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] PATH_INFO: `/v3/projects/8007d89ecd5d406995a16df4fc1b2c91` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:43.437400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8007d89ecd5d406995a16df4fc1b2c91)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:43.439446 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:43.527275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8007d89ecd5d406995a16df4fc1b2c91', 'request_id': 'req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:43.533816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8007d89ecd5d406995a16df4fc1b2c91', 'request_id': 'req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:43.595052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8007d89ecd5d406995a16df4fc1b2c91', 'request_id': 'req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:43.608165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] The token cache is being invalidate because project 8007d89ecd5d406995a16df4fc1b2c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:43.608453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:43.609159 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3ebc6f5b-9d97-4045-85a1-b3e7349755f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:43.610853 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1261/2495] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:43 2022] DELETE /identity/v3/projects/8007d89ecd5d406995a16df4fc1b2c91 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:43.616295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:43.616857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:43.616857 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:43.637895 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:43.639172 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:43.701296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1235/2496] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:43.709076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:43.709363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:43.709634 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:43.730407 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:43.731614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:43.739535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1262/2497] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:43.744274 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:43.754683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f9020-de19-4461-a8af-d9502ebb9c0b None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:43.760250 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:43.761067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:43.761346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:43.761616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:43.762261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:43.762560 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:43.764160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:43.776586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1236/2498] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:43.781339 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:43.790771 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e8b7-f607-4966-9f0d-59a55b37943f None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:43.795510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:43.796108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:43.796385 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:43.796654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:43.797319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:43.799212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:43.905253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1263/2499] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:43 2022] POST /identity/v3/projects => generated 399 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:43.911283 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:43.922079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95572716-a90a-438c-853a-93028607b1b3 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:43.981051 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:43.981656 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:43.981931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:43.982197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:43.982813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:43.984713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.092509 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1237/2500] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:43 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:44.096632 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.108485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aee983-cf7e-4baa-bf5d-993c3ff97314 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.113425 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.114065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.114345 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.114616 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.115274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.116917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.128375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1264/2501] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:44.132607 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.143612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906773-86ea-4ac3-818f-a9c8b0f9f58b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.148648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d4d44-31d8-413c-9eab-045c23cba44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.149315 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4d44-31d8-413c-9eab-045c23cba44c admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.149627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4d44-31d8-413c-9eab-045c23cba44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.150394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4d44-31d8-413c-9eab-045c23cba44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.151047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d4d44-31d8-413c-9eab-045c23cba44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.152686 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d4d44-31d8-413c-9eab-045c23cba44c admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.164418 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1238/2502] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:44.169729 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.180233 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a0ef3-820d-40be-a948-6f218ecca8e9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.187186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.187186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.187186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.187186 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] PATH_INFO: `/v3/projects/bc892b1e454347cc83b24dba4b84136d/users/8609a4f52c8e478499ee1b4717b6d6e9/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.203192 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc892b1e454347cc83b24dba4b84136d, user_id=8609a4f52c8e478499ee1b4717b6d6e9, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.205970 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.297813 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1265/2503] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:44 2022] PUT /identity/v3/projects/bc892b1e454347cc83b24dba4b84136d/users/8609a4f52c8e478499ee1b4717b6d6e9/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:44.309438 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.309750 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.310028 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.331918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:44.340360 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] MFA Rules not processed for user `8609a4f52c8e478499ee1b4717b6d6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:44.341785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:44.410614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1239/2504] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:44.415378 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.479278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463804-07ff-411e-9be8-b08b023e7d16 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.484848 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.485511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.485812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.486078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.486715 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.488607 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.493400 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:44.582931 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1266/2505] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:44 2022] POST /identity/v3/projects => generated 401 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:44.588397 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.599997 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27104d1d-9182-42b6-82bd-743e1b91a4b0 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.671789 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.672482 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.672605 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.672813 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.673481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.675481 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.798622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1240/2506] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:44 2022] POST /identity/v3/users => generated 307 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:44.803319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.814227 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc290c7-2bb4-49f1-94f9-22f1e248a5b5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.818800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.819399 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.819675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.819945 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.820605 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.823149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.833550 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1267/2507] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:44.837634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.847836 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215349bb-a0a1-4dd9-8b1f-12989f7cf31d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:44.852459 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:44.853214 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:44.853484 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:44.853757 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] PATH_INFO: `/v3/projects/a02f55271fa7470fa70d584cb08ef15a/users/5cb7cd55f19e4a6584817244b7ebde33/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:44.869181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02f55271fa7470fa70d584cb08ef15a, user_id=5cb7cd55f19e4a6584817244b7ebde33, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:44.872079 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:44.937634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1241/2508] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:44 2022] PUT /identity/v3/projects/a02f55271fa7470fa70d584cb08ef15a/users/5cb7cd55f19e4a6584817244b7ebde33/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:51:44.941610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:44.952723 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251294e-f03a-4dff-9c52-4bf8f8f55fe4 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:45.010334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796588da-2e91-4a25-9d10-fae6c7566d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:45.011007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796588da-2e91-4a25-9d10-fae6c7566d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:45.011287 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796588da-2e91-4a25-9d10-fae6c7566d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:45.011560 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796588da-2e91-4a25-9d10-fae6c7566d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:45.012196 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796588da-2e91-4a25-9d10-fae6c7566d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:45.013816 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796588da-2e91-4a25-9d10-fae6c7566d70 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:45.024093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1268/2509] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:45.028265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:45.038830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9e395-df14-4bdb-ac2d-3a9e936125b7 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:45.043966 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:45.044609 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:45.044882 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:45.045191 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] PATH_INFO: `/v3/projects/a02f55271fa7470fa70d584cb08ef15a/users/5cb7cd55f19e4a6584817244b7ebde33/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:45.047042 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02f55271fa7470fa70d584cb08ef15a, user_id=5cb7cd55f19e4a6584817244b7ebde33, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:45.049797 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:45.060975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1242/2510] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:45 2022] PUT /identity/v3/projects/a02f55271fa7470fa70d584cb08ef15a/users/5cb7cd55f19e4a6584817244b7ebde33/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:45.071484 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:45.071899 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:45.072268 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:45.101283 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] MFA Rules not processed for user `5cb7cd55f19e4a6584817244b7ebde33`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:45.102491 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:45.172479 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1269/2511] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:45.185744 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:45.187449 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:45.246719 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099f103-0964-41c7-979b-c2492980e7c9 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:45.250326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:45.250975 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:45.251255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:45.251526 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:45.258606 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:45.260905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:45.262391 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:45.271864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1243/2512] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:45.535133 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:45.545877 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc1c5-ff4a-4756-bc34-50510093a2c5 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:45.549449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783065dc-a1c5-4739-8ac3-f8994320b3e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:45.550057 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783065dc-a1c5-4739-8ac3-f8994320b3e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:45.550335 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783065dc-a1c5-4739-8ac3-f8994320b3e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:45.550690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783065dc-a1c5-4739-8ac3-f8994320b3e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:45.606457 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:45.610278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783065dc-a1c5-4739-8ac3-f8994320b3e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:45.618441 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783065dc-a1c5-4739-8ac3-f8994320b3e7 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:45.622529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1270/2513] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:50.804097 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:50.878627 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c26e1-b01f-42a0-8950-6f974f9d1351 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:50.883203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:50.883843 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:50.884131 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:50.884435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] PATH_INFO: `/v3/users/8609a4f52c8e478499ee1b4717b6d6e9` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:50.885643 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8609a4f52c8e478499ee1b4717b6d6e9)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:50.887786 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:51.083339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8609a4f52c8e478499ee1b4717b6d6e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:51.096176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8609a4f52c8e478499ee1b4717b6d6e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:51.182441 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8609a4f52c8e478499ee1b4717b6d6e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:51.183057 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8609a4f52c8e478499ee1b4717b6d6e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:51.183664 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8609a4f52c8e478499ee1b4717b6d6e9'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:51.198335 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1244/2514] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:50 2022] DELETE /identity/v3/users/8609a4f52c8e478499ee1b4717b6d6e9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:51.203064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:51.203531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:51.203662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:51.223397 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:51.224589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:51.284427 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1271/2515] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:51.294474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:51.295891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:51.356417 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca28c-e6f2-4e05-a9a6-2087dd89a99e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:51.360065 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:51.361394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:51.361394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:51.361394 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:51.371621 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:51.373093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:51.382826 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1245/2516] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:51.835531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:51.837821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:51.850248 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2e9b-b694-4a9e-ad9b-15c1dff7e81c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:51.855207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:51.855800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:51.856081 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:51.856355 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] PATH_INFO: `/v3/projects/bc892b1e454347cc83b24dba4b84136d` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:51.857749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc892b1e454347cc83b24dba4b84136d)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:51.859660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:51.990535 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc892b1e454347cc83b24dba4b84136d', 'request_id': 'req-31c3f74c-a042-4484-a357-c0a9ba63e5a5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:51.997925 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc892b1e454347cc83b24dba4b84136d', 'request_id': 'req-31c3f74c-a042-4484-a357-c0a9ba63e5a5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.090753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc892b1e454347cc83b24dba4b84136d', 'request_id': 'req-31c3f74c-a042-4484-a357-c0a9ba63e5a5'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.102452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] The token cache is being invalidate because project bc892b1e454347cc83b24dba4b84136d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:52.102751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.103391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.104897 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1272/2517] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:51 2022] DELETE /identity/v3/projects/bc892b1e454347cc83b24dba4b84136d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:52.109485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:52.120768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf8c3-83a9-4ae5-8ff8-b7dcd82df3d1 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:52.136304 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:52.186147 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:52.186831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:52.187105 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:52.187376 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] PATH_INFO: `/v3/users/5cb7cd55f19e4a6584817244b7ebde33` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:52.188604 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb7cd55f19e4a6584817244b7ebde33)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:52.190603 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:52.476493 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb7cd55f19e4a6584817244b7ebde33'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.488670 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb7cd55f19e4a6584817244b7ebde33'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.616973 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb7cd55f19e4a6584817244b7ebde33'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.617626 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb7cd55f19e4a6584817244b7ebde33'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.618339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-b0d77ee3-944d-4461-bf08-5daa28b84e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb7cd55f19e4a6584817244b7ebde33'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:52.632040 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1246/2518] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:51:52 2022] DELETE /identity/v3/users/5cb7cd55f19e4a6584817244b7ebde33 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:51:53.065032 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.066618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:53.079868 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3f74c-a042-4484-a357-c0a9ba63e5a5 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:53.139510 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:53.140162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.140424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.140698 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] PATH_INFO: `/v3/projects/a02f55271fa7470fa70d584cb08ef15a` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.141862 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02f55271fa7470fa70d584cb08ef15a)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:53.143757 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:53.168134 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:53.252498 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02f55271fa7470fa70d584cb08ef15a', 'request_id': 'req-b79b1698-8004-4ae2-8ceb-30930910c183'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:53.259474 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02f55271fa7470fa70d584cb08ef15a', 'request_id': 'req-b79b1698-8004-4ae2-8ceb-30930910c183'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:53.375039 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02f55271fa7470fa70d584cb08ef15a', 'request_id': 'req-b79b1698-8004-4ae2-8ceb-30930910c183'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:53.387303 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] The token cache is being invalidate because project a02f55271fa7470fa70d584cb08ef15a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81803) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:51:53.387599 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:53.388257 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-b79b1698-8004-4ae2-8ceb-30930910c183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:51:53.389942 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1273/2519] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:51:53 2022] DELETE /identity/v3/projects/a02f55271fa7470fa70d584cb08ef15a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:53.394838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.395353 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.395544 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.416614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:53.417871 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:53.482261 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1247/2520] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:53.490236 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.490689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.491064 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.512932 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:53.514320 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:53.522554 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1274/2521] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:51:53.528273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.538121 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691315e-3e32-44e2-9587-33bc239441d3 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:53.542939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:53.543552 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.543828 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.544099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] PATH_INFO: `/v3/domains` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.544748 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81802) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:51:53.545078 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:53.546715 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:53.563707 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1248/2522] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:51:53.565089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.575952 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26b233-7fd9-440a-8a8c-17b6376ae56c None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:53.580917 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:53.584669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.584669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.584669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.584669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:53.584669 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:53.741436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1275/2523] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:53 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:53.742698 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.753585 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db298ce-8d74-4e8c-8abf-7fc8031cd859 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:53.776437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:53.815646 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:53.816275 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.816550 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.816821 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.817490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:53.819414 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:53.914192 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1249/2524] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:53 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:53.920661 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.931010 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c6ba7-5bdf-48d1-a792-5f4193701ef8 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:53.935880 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:53.936583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.936861 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.937161 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.937802 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:53.939382 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:53.950440 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1276/2525] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:53.954476 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.965364 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1031b3f-de13-4487-a990-1497a95d809b admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:53.970083 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:53.970681 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:53.970957 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:53.971298 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:53.971940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:53.973599 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:53.984570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1250/2526] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:51:53.988728 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:53.999531 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc140b-f3d1-4e2b-a17b-54a26d7e7a02 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.004141 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f02ad-2a1f-4308-bc4a-aed79bd474e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.004785 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f02ad-2a1f-4308-bc4a-aed79bd474e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.005098 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f02ad-2a1f-4308-bc4a-aed79bd474e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.005368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f02ad-2a1f-4308-bc4a-aed79bd474e1 admin admin] PATH_INFO: `/v3/projects/ce3a383f0e5e4ac781db72cfd2760d33/users/9aae8cfb7b614516ad583b45038d488c/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.021221 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f02ad-2a1f-4308-bc4a-aed79bd474e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce3a383f0e5e4ac781db72cfd2760d33, user_id=9aae8cfb7b614516ad583b45038d488c, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.024000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f02ad-2a1f-4308-bc4a-aed79bd474e1 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.122630 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1277/2527] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:53 2022] PUT /identity/v3/projects/ce3a383f0e5e4ac781db72cfd2760d33/users/9aae8cfb7b614516ad583b45038d488c/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:54.134284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:54.135056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.135344 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.135610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.136905 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:54.175093 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] MFA Rules not processed for user `9aae8cfb7b614516ad583b45038d488c`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:54.176333 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:54.231282 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538986-7ecd-4699-a3cc-3d9270933c60 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.273485 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1278/2528] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:54.277901 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:54.307587 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.308206 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.308485 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.308758 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.310682 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:54.317347 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:54.324616 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.326289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.338201 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1251/2529] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:54.349610 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3d3b0-2464-49f3-8d1e-58ca7d2bcd38 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.355291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.356193 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.356472 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.356738 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.357488 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.359324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.479442 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1279/2530] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:54 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:54.484290 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:54.500400 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858d58-47a7-4b78-927b-dd1f1a7c79ec service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.530934 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:54.578366 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.579025 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.579307 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.579638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.580373 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.583330 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.691533 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1252/2531] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:54 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:54.698234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:54.718966 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337838e0-8ae9-4718-a370-d21e952ef59d admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.732311 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.733363 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.733836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.734404 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.735710 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.739185 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.766302 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1280/2532] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:54.767692 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:54.782617 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2ef8-9dd2-4202-be69-be611142055f admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.788073 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b1e2c-aa9e-4692-baa9-bf8cb97779da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.788691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b1e2c-aa9e-4692-baa9-bf8cb97779da admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.788993 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b1e2c-aa9e-4692-baa9-bf8cb97779da admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.789292 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b1e2c-aa9e-4692-baa9-bf8cb97779da admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.790324 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b1e2c-aa9e-4692-baa9-bf8cb97779da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.792546 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b1e2c-aa9e-4692-baa9-bf8cb97779da admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.804625 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1253/2533] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:54.807391 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:54.818801 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1f28-e9e7-45f2-8733-2c0eac8c1980 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:54.824430 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:54.825226 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.825576 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.825919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] PATH_INFO: `/v3/projects/0dbadcbcd8a14e428b5e1d8c6a8706e2/users/c8668b36a8154dc69a939a5f506e8e37/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.844329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbadcbcd8a14e428b5e1d8c6a8706e2, user_id=c8668b36a8154dc69a939a5f506e8e37, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:54.846164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:54.920201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1281/2534] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:54 2022] PUT /identity/v3/projects/0dbadcbcd8a14e428b5e1d8c6a8706e2/users/c8668b36a8154dc69a939a5f506e8e37/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:54.930921 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:54.931239 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:54.931508 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:54.963038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] MFA Rules not processed for user `c8668b36a8154dc69a939a5f506e8e37`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:54.964284 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:55.034877 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1254/2535] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:55.039436 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.106342 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4704c-b284-4a0e-b713-7333ceff0669 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.111122 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.111736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.112131 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.112308 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.112914 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:55.114956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:55.234053 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1282/2536] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:51:55 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:55.235840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.247467 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95971b06-b444-4dd3-a272-9403367d6c33 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.308638 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.309286 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.309564 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.309833 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] PATH_INFO: `/v3/users` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.310462 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:55.312406 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:55.427247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1255/2537] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:51:55 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:51:55.431793 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.442764 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d818c-628d-4acd-a2fa-687fbbc272cb admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.447876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.448506 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.448782 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.449086 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.449760 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:55.451548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:55.463640 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1283/2538] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:55.468203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.479377 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57dd46-2b4f-439e-844f-d5471c3b274a admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.485555 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.486168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.486412 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.486683 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] PATH_INFO: `/v3/projects/b9df30b0a48b42949b180258891b96c7/users/71a1776cbb4144ba9aa1d161a9a9b7c1/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.502919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9df30b0a48b42949b180258891b96c7, user_id=71a1776cbb4144ba9aa1d161a9a9b7c1, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:55.505611 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:55.577187 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:55.581887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1256/2539] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:55 2022] PUT /identity/v3/projects/b9df30b0a48b42949b180258891b96c7/users/71a1776cbb4144ba9aa1d161a9a9b7c1/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:55.585961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.596800 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604ab31-0023-4304-bcff-189e119a5aea admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.648095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:51:55.654845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590aa85-ee29-48ee-acec-be70efbcad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.655434 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590aa85-ee29-48ee-acec-be70efbcad74 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.655719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590aa85-ee29-48ee-acec-be70efbcad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.655991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590aa85-ee29-48ee-acec-be70efbcad74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.656665 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590aa85-ee29-48ee-acec-be70efbcad74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:55.658297 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590aa85-ee29-48ee-acec-be70efbcad74 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:55.668905 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1284/2540] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:51:55.673371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.684615 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ce0c-fcd7-4200-b34d-0492b01ff6b6 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.689488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.690149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.690149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.690149 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] PATH_INFO: `/v3/projects/b9df30b0a48b42949b180258891b96c7/users/71a1776cbb4144ba9aa1d161a9a9b7c1/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.692688 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9df30b0a48b42949b180258891b96c7, user_id=71a1776cbb4144ba9aa1d161a9a9b7c1, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:55.695488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:55.707170 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1257/2541] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:51:55 2022] PUT /identity/v3/projects/b9df30b0a48b42949b180258891b96c7/users/71a1776cbb4144ba9aa1d161a9a9b7c1/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:51:55.722089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.722368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.722646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:55.756316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] MFA Rules not processed for user `71a1776cbb4144ba9aa1d161a9a9b7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:51:55.757811 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:55.841568 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1285/2542] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:51:55.854969 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:55.856864 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:51:55.938519 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d8271-32ed-4544-b8b6-65699ff93991 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:55.943130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:55.943910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:55.944181 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:55.944455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:56.011768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:56.013177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:56.023920 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1258/2543] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:58.061709 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:58.072078 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1b2b5-6faa-45d3-8327-b4d4b70fd0d0 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:58.075629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:58.076230 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:58.076499 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:58.076765 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:58.137184 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:58.138668 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:58.148260 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1286/2544] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:51:59.930911 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:51:59.942168 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e47aa-ce31-42bb-b6f2-9d2e650a7009 service cinder] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:51:59.943841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37eb3-a076-4c7d-ace3-a68015c07a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:51:59.944472 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37eb3-a076-4c7d-ace3-a68015c07a0f service cinder] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:51:59.944767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37eb3-a076-4c7d-ace3-a68015c07a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:51:59.945067 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37eb3-a076-4c7d-ace3-a68015c07a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:51:59.954399 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c37eb3-a076-4c7d-ace3-a68015c07a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:51:59.955824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c37eb3-a076-4c7d-ace3-a68015c07a0f service cinder] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:51:59.965841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1259/2545] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:52:04.892820 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:04.976585 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96599b65-bf3a-499b-a7be-9b7151b3671c service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:04.981358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:04.981988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:04.982269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:04.982540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] PATH_INFO: `/v3/users/9aae8cfb7b614516ad583b45038d488c` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:04.983654 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aae8cfb7b614516ad583b45038d488c)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:04.985602 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:05.008203 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:05.310314 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aae8cfb7b614516ad583b45038d488c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:05.437989 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aae8cfb7b614516ad583b45038d488c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:05.452274 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aae8cfb7b614516ad583b45038d488c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:05.453038 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aae8cfb7b614516ad583b45038d488c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:05.453646 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aae8cfb7b614516ad583b45038d488c'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:05.467456 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1287/2546] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:52:04 2022] DELETE /identity/v3/users/9aae8cfb7b614516ad583b45038d488c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:05.472212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:05.472788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:05.472788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:05.494492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:52:05.495940 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:05.563946 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1260/2547] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:52:05.574569 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:05.575988 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:05.639737 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab179b-0a62-4a05-ad59-4e27dc40867c admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:05.643309 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:05.644006 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:05.644281 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:05.644551 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:05.654195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:05.655660 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:05.665789 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1288/2548] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:52:06.230586 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:06.232020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:06.245909 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05168860-3246-4a76-abf5-a04da638f2f8 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:06.252523 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:06.253295 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:06.253772 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:06.254054 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] PATH_INFO: `/v3/projects/ce3a383f0e5e4ac781db72cfd2760d33` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:06.255870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce3a383f0e5e4ac781db72cfd2760d33)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:06.258038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:06.294777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:06.397022 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce3a383f0e5e4ac781db72cfd2760d33', 'request_id': 'req-3ce9c533-38fa-4b35-b152-31a56afbd683'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.403812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce3a383f0e5e4ac781db72cfd2760d33', 'request_id': 'req-3ce9c533-38fa-4b35-b152-31a56afbd683'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.409717 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce3a383f0e5e4ac781db72cfd2760d33', 'request_id': 'req-3ce9c533-38fa-4b35-b152-31a56afbd683'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.489989 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] The token cache is being invalidate because project ce3a383f0e5e4ac781db72cfd2760d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:52:06.490834 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.490999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.492455 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1261/2549] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:52:06 2022] DELETE /identity/v3/projects/ce3a383f0e5e4ac781db72cfd2760d33 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:06.496865 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:06.506619 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97246714-08a3-42db-8ffb-264e729724b5 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:06.565923 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:06.566559 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:06.566845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:06.567120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] PATH_INFO: `/v3/users/c8668b36a8154dc69a939a5f506e8e37` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:06.568249 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8668b36a8154dc69a939a5f506e8e37)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:06.570201 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:06.783040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:06.788889 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8668b36a8154dc69a939a5f506e8e37'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.885492 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8668b36a8154dc69a939a5f506e8e37'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.900902 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8668b36a8154dc69a939a5f506e8e37'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.901707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8668b36a8154dc69a939a5f506e8e37'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.902317 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8668b36a8154dc69a939a5f506e8e37'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:06.919452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1289/2550] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:52:06 2022] DELETE /identity/v3/users/c8668b36a8154dc69a939a5f506e8e37 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:07.417116 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:07.418673 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:07.432700 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9c533-38fa-4b35-b152-31a56afbd683 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:07.497985 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:07.498660 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:07.498887 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:07.499160 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] PATH_INFO: `/v3/projects/0dbadcbcd8a14e428b5e1d8c6a8706e2` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:07.500289 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbadcbcd8a14e428b5e1d8c6a8706e2)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:07.502265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:07.635165 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbadcbcd8a14e428b5e1d8c6a8706e2', 'request_id': 'req-d4a11ee9-3b45-4af2-b763-18b362f231cf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:07.641608 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbadcbcd8a14e428b5e1d8c6a8706e2', 'request_id': 'req-d4a11ee9-3b45-4af2-b763-18b362f231cf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:07.647222 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbadcbcd8a14e428b5e1d8c6a8706e2', 'request_id': 'req-d4a11ee9-3b45-4af2-b763-18b362f231cf'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:07.747684 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] The token cache is being invalidate because project 0dbadcbcd8a14e428b5e1d8c6a8706e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:52:07.747967 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:07.748581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:07.750099 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1262/2551] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:52:07 2022] DELETE /identity/v3/projects/0dbadcbcd8a14e428b5e1d8c6a8706e2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:07.754182 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:07.755601 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:07.769119 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359b1b7-129b-483d-995f-c00d7be02e3e admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:07.828253 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:07.828892 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:07.829206 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:07.829476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] PATH_INFO: `/v3/users/71a1776cbb4144ba9aa1d161a9a9b7c1` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:07.830584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a1776cbb4144ba9aa1d161a9a9b7c1)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:07.832480 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:08.052845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a1776cbb4144ba9aa1d161a9a9b7c1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.124766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a1776cbb4144ba9aa1d161a9a9b7c1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.138365 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a1776cbb4144ba9aa1d161a9a9b7c1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.139045 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a1776cbb4144ba9aa1d161a9a9b7c1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.139642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f038702f-2310-47ef-8921-a6297e90fa90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a1776cbb4144ba9aa1d161a9a9b7c1'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.152254 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1290/2552] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:52:07 2022] DELETE /identity/v3/users/71a1776cbb4144ba9aa1d161a9a9b7c1 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:52:08.690917 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:08.692379 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:08.705348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a11ee9-3b45-4af2-b763-18b362f231cf admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:08.752783 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:08.769634 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:08.770279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:08.770594 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:08.771020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] PATH_INFO: `/v3/projects/b9df30b0a48b42949b180258891b96c7` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:08.772338 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9df30b0a48b42949b180258891b96c7)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:08.775739 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:08.978619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9df30b0a48b42949b180258891b96c7', 'request_id': 'req-c87caa47-03f7-4215-aa2e-8718a3339e68'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.985188 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9df30b0a48b42949b180258891b96c7', 'request_id': 'req-c87caa47-03f7-4215-aa2e-8718a3339e68'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:08.990824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9df30b0a48b42949b180258891b96c7', 'request_id': 'req-c87caa47-03f7-4215-aa2e-8718a3339e68'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:09.091527 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] The token cache is being invalidate because project b9df30b0a48b42949b180258891b96c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:52:09.091863 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:09.092486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-c87caa47-03f7-4215-aa2e-8718a3339e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:09.094036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1263/2553] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:52:08 2022] DELETE /identity/v3/projects/b9df30b0a48b42949b180258891b96c7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:09.098961 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.099433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.099548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.122422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:52:09.123675 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:09.162270 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:09.189059 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1291/2554] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:52:09.198581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.198581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.198581 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.220732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:52:09.222118 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:09.231672 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1264/2555] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:52:09.241578 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:09.251856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9caa5-c6ac-4d0d-8557-a4044a169672 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:09.256753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:09.257463 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.257745 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.258022 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.259162 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:52:09.259467 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:09.261116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:09.273322 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1292/2556] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 14:52:09.281880 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:09.291540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a89df4-36ff-4260-84a9-914073e10dfc None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:09.296175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:09.296767 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.297071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.297340 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.297972 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:09.299891 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:09.452889 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1265/2557] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:52:09 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:52:09.457461 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:09.467930 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0ea33-f28e-488c-b107-fd04f70407f3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:09.525313 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:09.525839 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.526139 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.526409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.527044 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:09.528938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:09.649589 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1293/2558] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:52:09 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:52:09.653886 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:09.664830 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527b110-a7db-437b-bf8e-e9910ca4d388 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:09.669486 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:09.670098 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.670371 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.670809 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.671453 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:09.673094 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:09.687654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1266/2559] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:52:09.688876 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:09.700960 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac097ae-eba2-4a20-8715-a2b7420f2ddf admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:09.705841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89b746-adcd-4165-a15e-8ae1dc21a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:09.706452 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89b746-adcd-4165-a15e-8ae1dc21a5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.706731 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89b746-adcd-4165-a15e-8ae1dc21a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.707000 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89b746-adcd-4165-a15e-8ae1dc21a5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.707636 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89b746-adcd-4165-a15e-8ae1dc21a5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:09.709291 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89b746-adcd-4165-a15e-8ae1dc21a5a8 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:09.720052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1294/2560] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 14:52:09.724296 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:09.735158 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1c567-5a42-48a9-b38d-f85a4a103a82 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:09.740488 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:09.741185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.741492 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.741771 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] PATH_INFO: `/v3/projects/e6956385cb1846b8b02bc517a81ce1fa/users/4e30582f46214d05a85f2b0542e69ed7/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.761714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6956385cb1846b8b02bc517a81ce1fa, user_id=4e30582f46214d05a85f2b0542e69ed7, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:09.764388 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:09.908712 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1267/2561] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:52:09 2022] PUT /identity/v3/projects/e6956385cb1846b8b02bc517a81ce1fa/users/4e30582f46214d05a85f2b0542e69ed7/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:09.913919 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:09.914207 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:09.914978 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:09.936749 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:52:09.939097 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:10.005707 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1295/2562] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:52:10.015936 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:10.017704 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:10.077540 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2545c-c6ee-410c-aa57-b6d9d38bf304 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:10.081265 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a2c6-ec5b-4250-a92a-d95b55c6e4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:10.081898 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a2c6-ec5b-4250-a92a-d95b55c6e4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:10.082174 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a2c6-ec5b-4250-a92a-d95b55c6e4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:10.082444 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a2c6-ec5b-4250-a92a-d95b55c6e4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:10.092740 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880a2c6-ec5b-4250-a92a-d95b55c6e4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:10.094309 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880a2c6-ec5b-4250-a92a-d95b55c6e4f3 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:10.104025 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1268/2563] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:52:10.778284 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:10.781123 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:10.792724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1282b4-83c2-4291-9b8f-406026bac62a None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:10.796232 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:10.796830 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:10.797137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:10.797412 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:10.799195 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:10.811489 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:10.812938 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:10.822895 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1296/2564] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:52:11.789246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c596ee5-abee-4d24-8568-bcd808999a05 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:11.791163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c596ee5-abee-4d24-8568-bcd808999a05 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:11.791729 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c596ee5-abee-4d24-8568-bcd808999a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:11.833761 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:11.845840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2c596ee5-abee-4d24-8568-bcd808999a05 None None] MFA Rules not processed for user `4e30582f46214d05a85f2b0542e69ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:52:11.847096 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2c596ee5-abee-4d24-8568-bcd808999a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:11.948273 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1269/2565] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:52:11.959871 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:11.962016 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:12.013116 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:12.047893 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1364cdc-c4de-44d2-b8a4-05637bd02382 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:12.057343 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:12.058954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:12.058954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:12.058954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:12.067936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:12.069424 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:12.082190 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1297/2566] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 14:52:14.114117 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:14.114680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:14.114680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:14.135763 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] MFA Rules not processed for user `4e30582f46214d05a85f2b0542e69ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:52:14.136988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:14.144382 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1270/2567] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:52:19.281095 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:19.282755 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:19.298694 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5ba7-c4c3-47f1-ae02-60b1eff48752 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:19.303736 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:19.304390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:19.305319 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:19.306527 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] PATH_INFO: `/v3/users/6f0516ac4c6648f9b208a89e174da3a3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:19.308433 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f0516ac4c6648f9b208a89e174da3a3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:19.310518 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:19.558243 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f0516ac4c6648f9b208a89e174da3a3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:19.634250 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f0516ac4c6648f9b208a89e174da3a3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:19.647553 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f0516ac4c6648f9b208a89e174da3a3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:19.648275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f0516ac4c6648f9b208a89e174da3a3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:19.649164 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f0516ac4c6648f9b208a89e174da3a3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:19.662295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1298/2568] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:52:19 2022] DELETE /identity/v3/users/6f0516ac4c6648f9b208a89e174da3a3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:19.995824 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:20.007344 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee348c-fd65-4e75-b922-500a4b94d0e4 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:20.073545 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:20.074339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:20.074622 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:20.074897 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] PATH_INFO: `/v3/projects/990b7e753ad1450489668d0ca151736e` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:20.076474 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990b7e753ad1450489668d0ca151736e)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:20.078514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:20.197435 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990b7e753ad1450489668d0ca151736e', 'request_id': 'req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.204133 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990b7e753ad1450489668d0ca151736e', 'request_id': 'req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.209850 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990b7e753ad1450489668d0ca151736e', 'request_id': 'req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.330689 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] The token cache is being invalidate because project 990b7e753ad1450489668d0ca151736e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:52:20.330935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.331513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.333245 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1271/2569] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:52:19 2022] DELETE /identity/v3/projects/990b7e753ad1450489668d0ca151736e => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:20.337555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:20.339149 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:20.353478 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59bfb7-decb-4366-8233-06b114bb4e53 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:20.426401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:20.426401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:20.426401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:20.426401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] PATH_INFO: `/v3/users/6144492748a14024b7bbf7bfaac0d36e` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:20.426803 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6144492748a14024b7bbf7bfaac0d36e)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:20.427684 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:20.701509 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6144492748a14024b7bbf7bfaac0d36e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.703454 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:20.776165 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6144492748a14024b7bbf7bfaac0d36e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.789730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6144492748a14024b7bbf7bfaac0d36e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.790583 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6144492748a14024b7bbf7bfaac0d36e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.791194 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6144492748a14024b7bbf7bfaac0d36e'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:20.804177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1299/2570] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:52:20 2022] DELETE /identity/v3/users/6144492748a14024b7bbf7bfaac0d36e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:21.075119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:52:21.076720 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:52:21.091575 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4a35-dd3e-4f11-a405-95c72432f8a4 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:52:21.133349 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:21.173479 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:52:21.174163 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:52:21.174440 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:52:21.174713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] PATH_INFO: `/v3/projects/aa384131227c4307945d5b791eb17e8d` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:52:21.176162 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa384131227c4307945d5b791eb17e8d)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:52:21.178175 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:52:21.312751 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa384131227c4307945d5b791eb17e8d', 'request_id': 'req-f2b30009-e8e5-4da5-8385-0e997a38b24e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:21.319334 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa384131227c4307945d5b791eb17e8d', 'request_id': 'req-f2b30009-e8e5-4da5-8385-0e997a38b24e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:21.326445 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa384131227c4307945d5b791eb17e8d', 'request_id': 'req-f2b30009-e8e5-4da5-8385-0e997a38b24e'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:21.472963 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] The token cache is being invalidate because project aa384131227c4307945d5b791eb17e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:52:21.473337 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:21.473974 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-f2b30009-e8e5-4da5-8385-0e997a38b24e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:52:21.475513 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1272/2571] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:52:21 2022] DELETE /identity/v3/projects/aa384131227c4307945d5b791eb17e8d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:52:31.421916 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:52:31.797773 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:53:14.340177 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:53:14.341618 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:53:14.414913 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290c2ee-4e9c-4117-b60c-69af03a34089 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:53:14.481900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:53:14.482686 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:53:14.482984 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:53:14.483401 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:53:14.485209 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:53:14.497213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:53:14.498790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:53:14.508951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1300/2572] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:53:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:53:24.504501 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=58 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:53:25.816258 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:53:25.816804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:53:25.816804 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:53:25.837217 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] MFA Rules not processed for user `4e30582f46214d05a85f2b0542e69ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:53:25.838395 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:53:25.895183 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1273/2573] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:53:35.888115 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=25 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:54:32.918507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:32.919991 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:32.988511 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e777fcb-1487-44fb-8eaf-130e282c6d3a service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:32.993091 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:32.993719 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:32.994003 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:32.994275 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] PATH_INFO: `/v3/users/4e30582f46214d05a85f2b0542e69ed7` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:32.995449 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e30582f46214d05a85f2b0542e69ed7)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:32.998258 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:33.195157 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e30582f46214d05a85f2b0542e69ed7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.293354 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e30582f46214d05a85f2b0542e69ed7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.306383 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e30582f46214d05a85f2b0542e69ed7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.311451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e30582f46214d05a85f2b0542e69ed7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.311451 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-3444a8e2-8502-403c-be17-418fbc0e857d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e30582f46214d05a85f2b0542e69ed7'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.320515 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1301/2574] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:54:32 2022] DELETE /identity/v3/users/4e30582f46214d05a85f2b0542e69ed7 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:54:33.603339 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:33.615593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a0cb9-f143-491f-967f-a653183bd147 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:33.674203 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:33.674838 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:33.675129 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:33.675416 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] PATH_INFO: `/v3/projects/e6956385cb1846b8b02bc517a81ce1fa` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:33.676547 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6956385cb1846b8b02bc517a81ce1fa)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:33.678450 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:33.791119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6956385cb1846b8b02bc517a81ce1fa', 'request_id': 'req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.798122 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6956385cb1846b8b02bc517a81ce1fa', 'request_id': 'req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.803999 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6956385cb1846b8b02bc517a81ce1fa', 'request_id': 'req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.899249 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] The token cache is being invalidate because project e6956385cb1846b8b02bc517a81ce1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:54:33.899538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.900185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-89f635d4-a0ba-43c6-8d9d-ce23e028eaeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:54:33.901959 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1274/2575] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:54:33 2022] DELETE /identity/v3/projects/e6956385cb1846b8b02bc517a81ce1fa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:54:33.908007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:33.908007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:33.908007 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:33.928575 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:54:33.929812 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:33.990790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1302/2576] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 14:54:33.999106 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:33.999386 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:33.999654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.020038 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:54:34.021277 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:34.029788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1275/2577] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:54:34.034079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.043821 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88a69-67fa-4ca9-a333-fe554839b627 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.048645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.049269 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.049557 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.049828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] PATH_INFO: `/v3/domains` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.050471 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81803) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 14:54:34.050772 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.052403 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.064220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1303/2578] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 14:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 14:54:34.068518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.078246 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3245d1-0810-460f-ac32-22753d76a77c None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.086061 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.086657 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.086942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.087196 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.087831 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.089846 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.203870 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1276/2579] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:54:34 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:54:34.208255 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.219514 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebe040-1d40-4787-94ff-f6b55d0e6502 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.223089 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:54:34.275808 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.276406 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.276689 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.276956 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.277678 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.280307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.368212 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1304/2580] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:54:34 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 14:54:34.373247 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.383365 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ebead-007e-4b47-99e6-88c9c9d8b818 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.388991 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.389614 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.389904 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.390176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.390841 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.392465 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.402713 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1277/2581] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:54:34.406852 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.416732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38069b59-4fc5-41c5-8398-d53cbf07fd57 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.421417 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bb83c-86a4-4dfe-b6f7-8b5acdbd9015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.428695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bb83c-86a4-4dfe-b6f7-8b5acdbd9015 admin admin] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.428695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bb83c-86a4-4dfe-b6f7-8b5acdbd9015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.428695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bb83c-86a4-4dfe-b6f7-8b5acdbd9015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.428695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bb83c-86a4-4dfe-b6f7-8b5acdbd9015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.428695 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bb83c-86a4-4dfe-b6f7-8b5acdbd9015 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.435814 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1305/2582] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:54:34.440084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.449942 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a700060-a7d1-4cf7-942e-2305b65e306d admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.455359 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.455950 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] REQUEST_METHOD: `PUT` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.456224 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.456501 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] PATH_INFO: `/v3/projects/b7940a345a7c4ccba03eff0476e82d6c/users/e5e2a8a810804aceb641c1bd752f35e3/roles/95d84bb23f684a9da8785b860247f822` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.471288 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7940a345a7c4ccba03eff0476e82d6c, user_id=e5e2a8a810804aceb641c1bd752f35e3, role_id=95d84bb23f684a9da8785b860247f822)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.474161 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.573918 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1278/2583] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:54:34 2022] PUT /identity/v3/projects/b7940a345a7c4ccba03eff0476e82d6c/users/e5e2a8a810804aceb641c1bd752f35e3/roles/95d84bb23f684a9da8785b860247f822 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:54:34.579867 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b297831-d5db-46db-8b58-b2ae7a66e8b7 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.580120 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b297831-d5db-46db-8b58-b2ae7a66e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.580390 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b297831-d5db-46db-8b58-b2ae7a66e8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.599469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-9b297831-d5db-46db-8b58-b2ae7a66e8b7 None None] MFA Rules not processed for user `e928bcf912f045deb4e67843e0dbbcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:54:34.600706 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-9b297831-d5db-46db-8b58-b2ae7a66e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:34.660328 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1306/2584] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:54:34.670123 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:34.671856 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:34.704785 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:54:34.731510 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6643d-d935-4041-b98d-852d9ea8c87e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:34.735636 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:34.736248 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:34.737902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:34.737902 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:34.745925 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:34.748124 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:34.757610 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1279/2585] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:54:40.611566 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:40.612137 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:40.612251 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:40.649922 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] MFA Rules not processed for user `e5e2a8a810804aceb641c1bd752f35e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:54:40.651220 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:40.726453 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1307/2586] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:54:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:54:40.732020 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:40.742931 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7964d2c-e7af-4af3-b1df-4e50c03f9f56 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:40.748363 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:40.749100 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:40.750211 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:40.750408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] PATH_INFO: `/v3/projects` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:40.751242 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:40.753323 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:40.861960 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1280/2587] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 14:54:40 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:54:40.866068 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:40.877909 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456776c1-bb91-4f48-be2b-0f97e8ab9ef4 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:40.889733 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:54:40.939967 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:40.940584 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:40.940860 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:40.941421 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:40.942050 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:40.944108 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:41.026295 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1308/2588] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 14:54:40 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 14:54:41.029752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:41.040921 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191b23-a835-40e6-b708-4eab7785852e admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:41.045777 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:41.046389 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:41.046665 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:41.046939 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:41.047619 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:41.049884 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:41.061384 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1281/2589] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:54:41.065645 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:41.075766 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd972-3101-4b2d-b3ef-515aeba849e3 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:41.080759 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:41.081466 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:41.081724 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:41.081993 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] PATH_INFO: `/v3/projects/6303b67346264d0dbfe9669293173dda/users/a075c3d4cf8f48c3a557e2ffce242a11/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:41.100642 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6303b67346264d0dbfe9669293173dda, user_id=a075c3d4cf8f48c3a557e2ffce242a11, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:41.103422 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:41.180614 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1309/2590] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:54:41 2022] PUT /identity/v3/projects/6303b67346264d0dbfe9669293173dda/users/a075c3d4cf8f48c3a557e2ffce242a11/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:54:41.184711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:41.196130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe957584-f0cc-4efc-80a9-401d7d8334d2 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:41.260593 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a7979-a0b5-45ed-8990-7f080f1d2935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:41.261236 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a7979-a0b5-45ed-8990-7f080f1d2935 admin admin] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:41.261515 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a7979-a0b5-45ed-8990-7f080f1d2935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:41.261788 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a7979-a0b5-45ed-8990-7f080f1d2935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:41.262429 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a7979-a0b5-45ed-8990-7f080f1d2935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:41.264030 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a7979-a0b5-45ed-8990-7f080f1d2935 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:41.274799 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1282/2591] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 14:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 14:54:41.278954 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:41.291790 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb7de-fe7a-4780-841b-252de90274c7 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:41.297571 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:41.299324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:41.299324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:41.299324 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] PATH_INFO: `/v3/projects/6303b67346264d0dbfe9669293173dda/users/a075c3d4cf8f48c3a557e2ffce242a11/roles/98256b71f5e74eb59e9c2fbd7b8134b5` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:41.301020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6303b67346264d0dbfe9669293173dda, user_id=a075c3d4cf8f48c3a557e2ffce242a11, role_id=98256b71f5e74eb59e9c2fbd7b8134b5)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:41.303797 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:41.315746 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1310/2592] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 14:54:41 2022] PUT /identity/v3/projects/6303b67346264d0dbfe9669293173dda/users/a075c3d4cf8f48c3a557e2ffce242a11/roles/98256b71f5e74eb59e9c2fbd7b8134b5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 14:54:47.420768 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454beb-6d9e-44c8-a818-3695600d6db6 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:47.421357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454beb-6d9e-44c8-a818-3695600d6db6 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:47.421357 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454beb-6d9e-44c8-a818-3695600d6db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:47.455154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-a8454beb-6d9e-44c8-a818-3695600d6db6 None None] MFA Rules not processed for user `a075c3d4cf8f48c3a557e2ffce242a11`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:54:47.455154 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-a8454beb-6d9e-44c8-a818-3695600d6db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:47.535791 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1283/2593] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:54:47.544975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:54:47.546873 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:47.627828 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a332f5-7545-429b-8579-63d655babaa9 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:54:47.633626 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:54:47.634278 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:47.634549 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:47.634827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:47.697742 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:54:47.699469 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:54:47.710008 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1311/2594] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:54:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:54:52.129565 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:54:52.130071 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:54:52.130197 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:54:52.149648 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] MFA Rules not processed for user `e5e2a8a810804aceb641c1bd752f35e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:54:52.150873 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:54:52.158008 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1284/2595] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:54:57.705415 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:55:02.144130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:55:07.959043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:55:07.960429 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:55:08.018968 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139a478-0ce1-4410-aecc-a8ceb7577698 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:55:08.022540 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e8f2a80ea142b7acb78ea661b0e47d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:55:08.023234 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:55:08.023507 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:55:08.023773 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:55:08.026135 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:55:08.039529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:55:08.041586 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:55:08.052364 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1312/2596] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:55:09.115185 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:55:09.116792 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:55:09.185050 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542fde9-8d50-49c1-a40a-56df443eba5f None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:55:09.188988 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:55:09.189752 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:55:09.190037 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:55:09.190322 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:55:09.192076 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:55:09.208868 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:55:09.210383 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:55:09.222685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1285/2597] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:55:12.833163 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:55:12.842658 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800e49-6f06-4119-8918-2ce2e09b4779 service cinder] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:55:12.846416 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:55:12.847005 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:55:12.847280 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:55:12.847548 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:55:12.849256 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:55:12.859918 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:55:12.861359 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:55:12.869552 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1313/2598] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:55:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:55:19.217176 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=26 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:55:22.865845 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=32 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:55:26.443716 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:55:26.455840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb594a-5cf1-4895-bd48-720b3cb60fa2 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:55:26.461346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:55:26.461982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:55:26.462255 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:55:26.462525 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:55:26.464221 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:55:26.476570 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:55:26.478114 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:55:26.489478 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1286/2599] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:55:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:55:29.261128 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:55:29.269795 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cb48d-6509-4c08-82b9-de82af7f6845 service neutron] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:55:29.273985 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a18564-32f0-4020-8bbc-3e15b31bde24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:55:29.274579 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a18564-32f0-4020-8bbc-3e15b31bde24 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:55:29.274856 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a18564-32f0-4020-8bbc-3e15b31bde24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:55:29.275173 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a18564-32f0-4020-8bbc-3e15b31bde24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:55:29.284494 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a18564-32f0-4020-8bbc-3e15b31bde24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:55:29.286034 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a18564-32f0-4020-8bbc-3e15b31bde24 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:55:29.296444 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1314/2600] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:55:36.484314 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=6 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:55:39.292166 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=6 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:57:39.853695 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:57:39.892426 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5996413-822f-4656-bf7f-b5ae0d4fa773 service nova] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:57:39.896350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:57:39.896910 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:39.897216 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:39.897490 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:39.899212 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:57:39.934130 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:57:39.936654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4d6b4-55a5-44f9-9389-c53748bcb297 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:57:39.944714 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1287/2601] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:57:40.328168 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9541b-4002-4ea2-9224-09b1d02a1155 None None] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.329213 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9541b-4002-4ea2-9224-09b1d02a1155 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.329714 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9541b-4002-4ea2-9224-09b1d02a1155 None None] PATH_INFO: `/` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.331144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1315/2602] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:57:40.337257 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6de9c0-220a-4aa1-91aa-38e90e822b72 None None] REQUEST_METHOD: `POST` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.337538 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6de9c0-220a-4aa1-91aa-38e90e822b72 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.337806 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6de9c0-220a-4aa1-91aa-38e90e822b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.401612 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.auth.core [None req-2d6de9c0-220a-4aa1-91aa-38e90e822b72 None None] MFA Rules not processed for user `b981e25e0b8d46f298b05b261433de94`. Rule list: `[]` (Enabled: `True`). {{(pid=81802) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:57:40.403559 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-2d6de9c0-220a-4aa1-91aa-38e90e822b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:57:40.410632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1288/2603] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:57:40.423373 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9541b-4002-4ea2-9224-09b1d02a1155 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:57:40.435062 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9541b-4002-4ea2-9224-09b1d02a1155 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:57:40.438904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584bb8a-d94c-481a-9e5b-9d33a230cea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:57:40.439503 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584bb8a-d94c-481a-9e5b-9d33a230cea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.439781 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584bb8a-d94c-481a-9e5b-9d33a230cea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.440052 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584bb8a-d94c-481a-9e5b-9d33a230cea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.447307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584bb8a-d94c-481a-9e5b-9d33a230cea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:57:40.448730 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584bb8a-d94c-481a-9e5b-9d33a230cea4 service neutron] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:57:40.457329 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1316/2604] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:57:40.543084 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713fa9e-65aa-451d-88ed-df67af131a08 None None] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.543362 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713fa9e-65aa-451d-88ed-df67af131a08 None None] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.543632 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713fa9e-65aa-451d-88ed-df67af131a08 None None] PATH_INFO: `/` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.545064 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1289/2605] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 14:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 14:57:40.550555 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] REQUEST_METHOD: `POST` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.550841 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.554435 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.589144 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.auth.core [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] MFA Rules not processed for user `ea17166a0ace485dbf6cadaae7731ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81803) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 14:57:40.590334 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:57:40.620900 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1317/2606] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 14:57:40.631663 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713fa9e-65aa-451d-88ed-df67af131a08 None None] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:57:40.643177 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713fa9e-65aa-451d-88ed-df67af131a08 None None] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:57:40.646822 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b981e25e0b8d46f298b05b261433de94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:57:40.647432 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.647705 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.647970 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.658631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:57:40.660104 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:57:40.671068 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1290/2607] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:57:40.733368 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:57:40.734824 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:57:40.804033 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246872-b6ea-4d4f-823a-6fde70bdc495 None None] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:57:40.809101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:57:40.809740 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:57:40.810020 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:57:40.810395 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:57:40.812409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:57:40.824043 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:57:40.825690 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:57:40.835753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1318/2608] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:57:50.666328 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=50 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:57:50.831703 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=56 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:58:45.368514 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:58:45.369938 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:58:45.406346 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf907f-d760-4de2-a194-c9d0f7167932 service neutron] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:58:45.411348 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:58:45.411945 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:58:45.412220 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:58:45.412491 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:58:45.414190 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:58:45.426119 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:58:45.427685 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:58:45.437580 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1291/2609] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:58:55.433182 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=15 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:59:10.227389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:59:10.227389 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:59:10.245941 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52278b87-86e6-4906-824d-48f5a329ec73 service placement] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:59:10.254106 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea17166a0ace485dbf6cadaae7731ed6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:59:10.254826 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:59:10.255101 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:59:10.255513 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:59:10.257316 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:59:10.273591 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:59:10.275215 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:59:10.290202 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1319/2610] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:59:18.786431 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:59:18.794982 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef201eb4-f0c5-4c5a-a804-6151528ae8e1 service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:59:18.800780 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:59:18.801437 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] REQUEST_METHOD: `GET` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:59:18.801711 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:59:18.801978 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:59:18.803680 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:59:18.815654 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:59:18.818002 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:59:18.828043 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1292/2611] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 14:59:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 14:59:20.283361 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=6 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:59:22.888375 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:59:22.978975 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43fe3e-fa3b-4173-be55-d626b9d43bf7 service nova] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:59:22.983752 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:59:22.984381 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:59:22.984662 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:59:22.984936 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] PATH_INFO: `/v3/users/e5e2a8a810804aceb641c1bd752f35e3` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:59:22.986076 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e2a8a810804aceb641c1bd752f35e3)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:59:22.988023 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:59:23.194409 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e2a8a810804aceb641c1bd752f35e3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.276529 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e2a8a810804aceb641c1bd752f35e3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.291040 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e2a8a810804aceb641c1bd752f35e3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.291751 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e2a8a810804aceb641c1bd752f35e3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.292358 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e2a8a810804aceb641c1bd752f35e3'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.305037 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1320/2612] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:59:22 2022] DELETE /identity/v3/users/e5e2a8a810804aceb641c1bd752f35e3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:59:23.581754 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:59:23.593948 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea505-bfda-49fd-8eb0-4a2b9243d736 service placement] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:59:23.653641 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:59:23.654350 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:59:23.654631 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:59:23.654892 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] PATH_INFO: `/v3/projects/b7940a345a7c4ccba03eff0476e82d6c` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:59:23.656019 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7940a345a7c4ccba03eff0476e82d6c)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:59:23.658032 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:59:23.758279 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7940a345a7c4ccba03eff0476e82d6c', 'request_id': 'req-837c7a2e-bc7d-4224-845d-4834f4081475'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.764899 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7940a345a7c4ccba03eff0476e82d6c', 'request_id': 'req-837c7a2e-bc7d-4224-845d-4834f4081475'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.771047 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7940a345a7c4ccba03eff0476e82d6c', 'request_id': 'req-837c7a2e-bc7d-4224-845d-4834f4081475'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.860727 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] The token cache is being invalidate because project b7940a345a7c4ccba03eff0476e82d6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:59:23.861036 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.861709 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:23.866724 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1293/2613] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:59:23 2022] DELETE /identity/v3/projects/b7940a345a7c4ccba03eff0476e82d6c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:59:23.870696 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:59:23.872138 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:59:23.890371 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714bd4-e266-4b2b-94f4-32296241dbce admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:59:23.947079 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:59:23.947629 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:59:23.947904 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:59:23.948174 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] PATH_INFO: `/v3/users/a075c3d4cf8f48c3a557e2ffce242a11` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:59:23.952832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a075c3d4cf8f48c3a557e2ffce242a11)` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:59:23.952832 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:59:23.954307 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:59:24.290251 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a075c3d4cf8f48c3a557e2ffce242a11'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.361951 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a075c3d4cf8f48c3a557e2ffce242a11'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.375753 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a075c3d4cf8f48c3a557e2ffce242a11'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.376438 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a075c3d4cf8f48c3a557e2ffce242a11'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.377056 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.notifications [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a075c3d4cf8f48c3a557e2ffce242a11'} {{(pid=81803) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.389836 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1321/2614] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 14:59:23 2022] DELETE /identity/v3/users/a075c3d4cf8f48c3a557e2ffce242a11 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:59:24.620919 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Authenticating user token {{(pid=81802) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 14:59:24.622732 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.fernet_utils [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81802) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 14:59:24.637093 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c7a2e-bc7d-4224-845d-4834f4081475 admin admin] Validating token access rules against request {{(pid=81802) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 14:59:24.694518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e928bcf912f045deb4e67843e0dbbcf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e747be529418447d86ba59ccbdb4ada2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81802) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 14:59:24.695145 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 14:59:24.695422 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 14:59:24.695691 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] PATH_INFO: `/v3/projects/6303b67346264d0dbfe9669293173dda` {{(pid=81802) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 14:59:24.696795 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6303b67346264d0dbfe9669293173dda)` {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 14:59:24.698736 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] RBAC: Authorization granted {{(pid=81802) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 14:59:24.707518 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:59:24.825812 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6303b67346264d0dbfe9669293173dda', 'request_id': 'req-4d383734-969b-47b4-9dde-b88ebef433fe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.832916 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6303b67346264d0dbfe9669293173dda', 'request_id': 'req-4d383734-969b-47b4-9dde-b88ebef433fe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.838935 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6303b67346264d0dbfe9669293173dda', 'request_id': 'req-4d383734-969b-47b4-9dde-b88ebef433fe'} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.910408 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] The token cache is being invalidate because project 6303b67346264d0dbfe9669293173dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81802) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 14:59:24.910696 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.911326 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG keystone.notifications [None req-4d383734-969b-47b4-9dde-b88ebef433fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81802) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 14:59:24.913157 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: [pid: 81802|app: 0|req: 1294/2615] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 14:59:24 2022] DELETE /identity/v3/projects/6303b67346264d0dbfe9669293173dda => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 14:59:34.385732 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 14:59:34.906840 dv-cinder-862365-4-iscsi devstack@keystone.service[81802]: DEBUG dbcounter [-] [81802] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81802) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 15:03:50.589927 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Authenticating user token {{(pid=81803) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 15:03:50.591348 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 15:03:50.660804 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a033c-6f38-48da-b1bd-ebe2f4132a13 admin admin] Validating token access rules against request {{(pid=81803) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 15:03:50.720219 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a0aa7593164a57970b48b14c14a41a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca7b5445846e410da564b144f3341e91', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81803) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 15:03:50.720827 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] REQUEST_METHOD: `GET` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 15:03:50.721148 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] SCRIPT_NAME: `/identity` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 15:03:50.721476 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81803) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 15:03:50.723502 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.fernet_utils [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81803) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 15:03:50.734720 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 15:03:50.736293 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7ac1b-c276-4528-ace2-584cbdab1ca7 service placement] RBAC: Authorization granted {{(pid=81803) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 15:03:50.747065 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: [pid: 81803|app: 0|req: 1322/2616] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 15:04:00.742564 dv-cinder-862365-4-iscsi devstack@keystone.service[81803]: DEBUG dbcounter [-] [81803] Writing DB stats keystone:SELECT=58 {{(pid=81803) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}